How to prepare varied data for import: CSV, XLSX, JSON and NDJSON

Almost every import problem starts in the file: a different delimiter, a date in a local format, a number with a comma, a column with a swapped name. This guide walks through the four formats DeepMiner imports and shows what to check in each one before uploading.

Before the file, the schema

The schema says which fields the index has, the type of each one and which field identifies the record. Defining that first keeps you from importing data that does not serve the questions you are going to ask.

In DeepMiner, the fastest way is to paste a sample record in JSON and use Infer fields. The available types are text, number, date, boolean, object and list. Review each field’s type and mark the key field: with it, reimporting the same record updates instead of duplicating. Fields already saved become read-only, and new fields can be added later.

The column names in the file need to match the schema field names, spelled the same way. Names starting with an underscore are reserved.

CSV

CSV is the most common format and the one that varies most. Check:

• Header on the first line, with the field names.
• A single delimiter: semicolon or comma. If the header has a semicolon, DeepMiner uses semicolon for the whole file.
• No delimiter inside values. An address with a comma in a comma-separated CSV splits the line into extra columns. Prefer semicolon as the delimiter or remove the character from the value.
• One line per record, with no line breaks inside a field.
• UTF-8 encoding, so accents arrive correctly.

XLSX

The spreadsheet solves part of CSV’s problems, because each cell is already a separate value. The care goes into the structure: DeepMiner reads the first sheet, uses the first filled row as the header and imports the rows below it. Completely empty rows are ignored.

Before exporting, remove titles, total rows and merged cells above the header, and keep the data on the first sheet.

JSON and NDJSON

When the record has structure, such as a customer with a list of orders, JSON and NDJSON preserve that structure without flattening columns. The index schema accepts objects and lists of objects, and the fields inside become nested fields.

JSON must be an array of objects, each object one record. NDJSON has one complete object per line, with no comma between lines and no brackets around them. If a line in the NDJSON is invalid, the import reports the line number.

Both formats follow the public specifications cited in the sources. Files generated by standard libraries are usually correct. The ones written by hand are the ones that fail most.

Dates, numbers and booleans

Values are validated by the field’s type. Three formats prevent most invalid records:

• Dates as YYYY-MM-DD, such as 2025-01-31. Local formats can be read with day and month swapped.
• Numbers with a period as the decimal separator and no thousands separator: 1234.56.
• Booleans as true or false, 1 or 0, yes or no.

A record with an invalid value in a field is imported with the invalid mark and the error message, so you know which field to fix. The key field cannot be empty.

After importing

The import screen shows the total records, the processed ones, the failures and the status, with progress updating by itself. The Import History keeps each file, the source (file or API) and the error report.

Check a few records in the index before using the data in flows. If something came in wrong, fix the file and import again: with the key field, the records are updated.

Want to import a sample of your data in the demo? Book it and bring a file with fictional data.