JSON

@talosjs/json reads a JSON file as a stream, handing you one top-level object at a time through an async generator. That’s the whole point: you can walk a multi-gigabyte array without ever holding it all in memory. The Json class can also drop records that match a pattern and write what’s left to YAML or CSV.

Installation

Install the package with Bun.

bun add @talosjs/json

Usage

Construct a Json instance with a file path, optionally typing the records, then iterate with for await. Each top-level object (or array element) is parsed and yielded one at a time.

import { Json } from "@talosjs/json";

type User = { id: number; name: string; email: string };

const json = new Json<User>("./users.json");

for await (const user of json.load()) {
  console.log(user.id, user.name);
}

Ignoring records

Pass an ignore map of field-to-RegExp. Any record whose field matches its pattern is skipped during iteration.

import { Json } from "@talosjs/json";

type User = { id: number; name: string; email: string };

const json = new Json<User>("./users.json");

// Skip records whose email is a test/example address
for await (const user of json.load({ ignore: { email: /@example\.com$/ } })) {
  console.log(user.email);
}

Converting to YAML or CSV

toYaml and toCsv consume the same stream and write to an output path. toCsv requires the headers to emit and a separator; both accept the same ignore filter.

import { Json } from "@talosjs/json";

type User = { id: number; name: string; email: string };

const json = new Json<User>("./users.json");

await json.toYaml({ path: "./users.yaml" });

await json.toCsv({
  path: "./users.csv",
  headers: ["id", "name", "email"],
  separator: ",",
  ignore: { email: /@example\.com$/ },
});

You can read the source path back at any time with json.getPath().

When it’s worth reaching for

Reach for Json when the file is too big to parse in one go, or when you’d otherwise be writing the streaming and filtering plumbing yourself. load() streams records one at a time, the ignore map skips the ones you don’t want as they pass through, and toYaml/toCsv let you re-export without pulling in a separate serialization library. It fits ETL-style work nicely, where each record gets transformed on its way out. For a small config file or an API response, skip all of this. If the document already fits in memory, JSON.parse() is the simpler answer.

Getting Started

AI

Framework

Command Line

Installation

Usage

Ignoring records

Converting to YAML or CSV

When it’s worth reaching for

​Installation

​Usage

​Ignoring records

​Converting to YAML or CSV

​When it’s worth reaching for

Installation

Usage

Ignoring records

Converting to YAML or CSV

When it’s worth reaching for