Parsing a real-world CSV export: five assumptions that break

For a portfolio project I built a browser tool that imports a position-history CSV exported from a crypto exchange and rebuilds the chart around each trade. The parsing is maybe forty lines. It took far longer than forty lines usually takes, because a CSV that a real product generates for humans is not the CSV you learned to parse.

Every one of these bit me. None of them are exotic.

1. The header row is not the first row

Exports written for humans put a metadata block above the table — account name, export date, display timezone, filters used. The column headers might be on line 4, or line 7, and the number isn’t stable across versions.

So don’t index. Search for the header by content:

const hi = lines.findIndex(
  (l) => l.includes("开仓均价") && l.includes("平仓均价")
);
if (hi < 0) return null;

Two column names, not one, and deliberately so. A single distinctive word can easily appear in the metadata block above — a filter description mentioning one column name would match and send everything after it off by several rows. Requiring two names that only co-occur in a real header makes a false positive very unlikely.

If neither is found, bail out immediately and tell the user the file isn’t recognised. A parser that limps on after failing to find its header produces garbage that looks like data, which is much worse than an error.

2. There’s an invisible character at the start of the file

UTF-8 files often begin with a byte order mark. It’s zero-width, so the file looks completely normal in every editor — and it gets glued onto the front of the very first field.

Which means this fails:

hdr.indexOf("仓位创建时间")   // -1, because the actual string is "仓位创建时间"

You end up staring at two strings that render identically and comparing as unequal. Strip it on the way in:

const lines = text.split(/\r?\n/).map((l) => l.replace(//g, ""));

Note the /\r?\n/ too. Files exported from Windows tools carry \r\n, and splitting on \n alone leaves a trailing \r on every line — so your last column of every row has an invisible character on the end, and that comparison silently fails instead.

Two invisible characters, two silent failures, four lines of defence.

3. Column positions are not stable — look them up by name

The tempting version is row[0], row[3], row[7]. It works until the exporter adds a column, and then every field is shifted and nothing throws — you just get prices where the leverage should be.

const hdr = lines[hi].split(",");
const ci = (n) => hdr.indexOf(n);
const c = {
  ct: ci("仓位创建时间"), ut: ci("仓位更新时间"),
  inst: ci("交易产品"),   dir: ci("持仓方向"),
  open: ci("开仓均价"),   close: ci("平仓均价"),
  pnl: ci("收益额"),      pr: ci("收益率"),
};

if (c.open < 0 || c.close < 0 || c.inst < 0) return null;

Resolve names to indices once, then use the map. And check that the columns you actually require were foundindexOf returns -1 for a missing column, and row[-1] is undefined, which will propagate quietly through parseFloat into NaN and surface much later as a blank chart.

Failing at the point of the missing column is the whole game. NaN is a terrible error message.

4. Not every line is a data row

Real exports have blank lines, sometimes a totals row at the bottom, sometimes a truncation notice. One bad line should not take down the import.

for (let i = hi + 1; i < lines.length; i++) {
  const r = lines[i].split(",");
  if (!r[c.inst] || !r[c.open]) continue;

  const op = parseFloat(r[c.open]);
  if (!Number.isFinite(op)) continue;

  const ros = rawSec(r[c.ct]), rcs = rawSec(r[c.ut]);
  if (ros == null || rcs == null) continue;
  // ...
}

Skip rows that don’t have the fields that make them meaningful. Number.isFinite rather than !isNaN, because isNaN("") is false — an empty string coerces to 0 and sails through. Number.isFinite also rejects Infinity, which is what you get from a malformed numeric field.

Then, at the end, one check that matters:

if (!trades.length) return null;

Successfully parsing zero rows is not success. If the loop skipped everything, the format assumption was wrong, and the user needs to hear that rather than receive an empty screen.

5. new Date(string) is not a parser

This is the one I’d most want someone to take away.

new Date("2025-03-14 09:35:00")

What this returns depends on the browser and on the machine’s local timezone. A date-time string without an offset gets interpreted as local time in most engines — so the same file parsed by two users in two countries produces two different instants. For a tool whose entire job is to line trades up against market candles, that’s fatal.

Parse it explicitly instead:

const rawSec = (s) => {
  const m = String(s).trim()
    .match(/(\d{4})-(\d{2})-(\d{2})[ T](\d{2}):(\d{2})(?::(\d{2}))?/);
  if (!m) return null;
  return Math.floor(
    Date.UTC(+m[1], +m[2] - 1, +m[3], +m[4], +m[5], +(m[6] || 0)) / 1000
  );
};
  • Date.UTC rather than the Date constructor — no local-timezone interpretation, ever. The result is deterministic on every machine.
  • [ T] accepts both 2025-03-14 09:35 and 2025-03-14T09:35, because exports are inconsistent about this.
  • (?::(\d{2}))? makes seconds optional — some exports include them, some don’t.
  • Returns null on no match, which is what feeds the row-skip in point 4.
  • +m[2] - 1 because Date.UTC takes months 0-indexed. Every date bug’s favourite hiding place.

The subtlety here: the timestamps come out as UTC deliberately, and wrongly. The file’s real offset isn’t known at this stage, so the parser records the wall-clock reading as if it were UTC and keeps those raw seconds so the offset can be applied later, once it’s been determined. Parse first, interpret second — don’t bake a guess into the parse step.

The thing I’d fix

line.split(",") is not CSV parsing. It breaks on any quoted field containing a comma:

"Widget, Large",42,1.5

This particular export has no free-text columns, so it holds — but it’s an assumption about someone else’s format, and formats change. If the export ever gains a note or description column, this parser corrupts every row after it, silently. For anything user-facing and long-lived, use a real CSV library.

I’m keeping it because the constraint is real and documented, not because it’s correct. Those are different things, and it’s worth being honest about which one you’re relying on.

Takeaway

The common thread is that every one of these produces wrong data rather than an error. A BOM doesn’t throw. A shifted column doesn’t throw. new Date() doesn’t throw — it confidently returns a time that’s wrong by hours. That’s why the defence is mostly checking rather than catching: verify the header exists, verify required columns resolved, verify numbers are finite, verify at least one row survived.

When you’re parsing a file you don’t control, assume it will change under you, and make it fail loudly when it does.


This is a portfolio project, not commercial software, and describes no trading activity or results. Built with AI-assisted development: I define requirements and acceptance criteria, iterate on the implementation, then review, test and debug it.

Discover more from ZFLI Works

Subscribe now to keep reading and get access to the full archive.

Continue reading