Lots of CSVs

Lots of CSVs (lotsofcsvs) is a small data warehouse built with CSV and HTTP as its interface. It is intentionally as barebones as possible. It aims to be a simple API for recording and fetching data that works with everything.

The original motivation for it was to create a data API that you could use from iOS Shortcuts. Here is a template shortcut that you can modify.

# Documentation: The API
#
# Get a dataset as CSV
# dataset/names/can/be/paths/like/this.csv so you can use them for organization
curl -H "authorization: Bearer <token>" /api/u/user_name/path/to/cool_data.csv
# Get a dataset as a SQLite database
# The table is named after the dataset: SELECT * FROM "path/to/cool_data"
curl -H "authorization: Bearer <token>" -o cool_data.sqlite \
 /api/u/user_name/path/to/cool_data.sqlite
# Append to the dataset
# The .csv suffix is optional and you can use paths to name your datasets.
cat example.csv | curl -X POST -H "Content-Type: text/csv; header=absent"
 -H "authorization: Bearer <token>" \
 https://www.lotsofcsvs.com/api/u/<your_username>/totally/going/to/delete/example --data-binary @-
# You can post as many rows as you want.
# Use the header=present (or header=absent) variable in the mimetype.
# When in doubt follow https://www.rfc-editor.org/rfc/rfc4180
# If you are appending to a dataset that doesn't exist it will be created.
# You must send the CSV headers the first time you post to it.
# Otherwise lotsofcsvs has to guess at the names and we're not doing that.
#
# If you need a CSV to test with, we have one.
curl -o example.csv https://www.lotsofcsvs.com/example.csv
cat example.csv | curl -X POST -H "Content-Type: text/csv; header=present" \
 -H "authorization: Bearer <token>" \
 https://www.lotsofcsvs.com/api/u/<your_username>/example.csv --data-binary @-
# Bulk import a CSV file
# For large CSV files, use the bulk import API which processes files asynchronously
curl -X POST -H "authorization: Bearer <token>" \
 -F "file=@/path/to/your/file.csv" \
 -F "datasetName=cool_data" \
 -F "isNewDataset=true" \
 https://www.lotsofcsvs.com/api/import/upload
# This returns a jobId that you can use to check the import status:
curl -H "authorization: Bearer <token>" \
 https://www.lotsofcsvs.com/api/import/status?jobId=<jobId>
# Exporting your whole account
# It is built in the background, so the first call - and the first call after
# anything changed - answers 202 with Location and Retry-After: 10.
# Poll the same URL until it answers 200; polling while a job is running
# does not start a second one.
curl -i -H "authorization: Bearer <token>" https://www.lotsofcsvs.com/api/u/user_name.zip
{"status":"pending","format":"zip","location":"/api/u/user_name.zip","retry_after":10,"error":null}
# Poll until it is ready, then keep the file. Swap .zip for .tar.gz or .sqlite
# for the other two. Only 202 means "still building", so stop on anything else.
until code=$(curl -s -o user_name.zip -w '%{http_code}' \
 -H "authorization: Bearer <token>" \
 https://www.lotsofcsvs.com/api/u/user_name.zip); [ "$code" != 202 ]; do sleep 10; done
[ "$code" = 200 ] || { echo "not downloaded: HTTP $code"; cat user_name.zip; }
# The archives hold one CSV per dataset at csvs/user_name/<dataset>.csv;
# the database holds one table per dataset, named exactly after it.
# If the database cannot be built for your account, you still get both archives
# and .sqlite answers 404 with the reason.
# using DuckDB
duckdb
CREATE SECRET http (
        TYPE HTTP,
        EXTRA_HTTP_HEADERS MAP {
          'authorization': 'Bearer <token>'
          }
        );
CREATE TABLE example
 AS SELECT * FROM read_csv('https://www.lotsofcsvs.com/example.csv');
CREATE TABLE your_table_name
 AS SELECT * FROM read_csv('https://www.lotsofcsvs.com/api/u/<username>/your.csv');
# It is also LLM friendly. There is a prompt telling LLMs how to use this site
# hidden in this page. Also, if you programmatically fetch this page
# it will return the prompt, similar to https://docs.jina.ai/.
curl -L https://www.lotsofcsvs.com/.well-known/llms.txt
# Or run a Model Context Protocol server
curl -O https://www.lotsofcsvs.com/requirements.txt
curl -O https://www.lotsofcsvs.com/csv_mcp_server.py
pip install -r ./requirements.txt
python csv_mcp_server.py --url https://www.lotsofcsvs.com
# Or use the Remote MCP Server (built-in)
# lotsofcsvs.com has a built-in MCP server that uses OAuth 2.1 for authentication
# Step 1: Create an OAuth client
# Option A: Via web UI
1. Go to https://www.lotsofcsvs.com/u/<your_username>/profile
2. In the OAuth Clients section, click "Create OAuth Client"
3. Enter a name (e.g., "Claude Desktop")
4. Select scopes: mcp.read and mcp.ingest
5. Save your client_id and client_secret (shown only once!)
# Option B: Dynamic registration (RFC 7591)
curl -X POST https://www.lotsofcsvs.com/api/oauth/register \
 -H "Authorization: Bearer <your_session_token>" \
 -H "Content-Type: application/json" \
 -d '{"client_name":"My MCP Client","grant_types":["client_credentials"],"scope":"mcp.read mcp.ingest"}'