01 / 01
One Night on Nebius · 20 Aug 2026 · Frontier Tower Floor 10 + online
● session 1 of 2

Build your own dataset

Tonight is the data. / You leave with a file you wrote and a way to check it. Friday we train on it.

Rayyan Zahid · Nebius Fellow · Immersive Commons
> press → or space to advance
● 24 slides · notes on N
● A dataset built by hand

5,000,000
slips of paper

To build the Oxford English Dictionary, volunteers mailed in example sentences. One sentence per slip. The dictionary was built out of that pile.

Fig. 01/Oxford English Dictionary/Public call for readers, 1879. First section published 1884. Finished 1928.

● The same idea, in a file

Each slip held three things. So does each row of your dataset.

On a slip

  • The word being defined
  • One sentence using it correctly
  • Where that sentence came from

In a row

  • instruction — what someone asks
  • response — the answer you want back
  • context — extra material, optional

Fig. 02/Slip and row/You are collecting examples of the behaviour you want. That is all a dataset is.

● Tonight, 6 to 9

Tonight is the dataset. No GPU, no training.

01SetupWhat fine-tuning does and does not do, and what a row looks like · 30 min
02WriteThe main block. Your own file, from your own material · 75 min
03CheckDuplicates, empty rows, overlap with your test set · 20 min
04Test setWrite the questions you will judge Friday's result with · 20 min
05ShareCompare rows across the room. This is where most people fix theirs · 20 min
06FridayA walk through the training run your file goes into · 15 min

Session 2 is Friday 22 August, same time and place. That is the one with the GPUs. Bring the file you build tonight.

Anyone can download the same model. Your data is the part that is yours.

Same weights, same code, same GPU · the dataset is the only thing that differs

● Before you start

Fine-tuning changes how a model answers, not what it knows.

Use it for

  • Format: always answer in a particular structure
  • Tone: sound like your team instead of a generic assistant
  • Habits: the steps an expert takes without being told
  • Edge cases: what to do with a bad or hostile input

Do not use it for

  • Teaching facts the model has never seen. Use retrieval for that
  • Fixing a prompt you have not tried improving yet
  • Tasks you can simply describe in the system prompt
  • Making a small model as capable as a large one

Fig. 03/Scope/If a better prompt already fixes the problem, you do not need a dataset.

● The file format

One row is one line of JSON.

data/mine.jsonl one object per line · no commas between lines
{"instruction": "Summarise this bug report for the standup.",
"context": "Login fails on Safari 17 when the session cookie is...",
"response": "Safari 17 drops our session cookie. Auth team is on it. No workaround yet."}
 
{"instruction": "Write the release note for the caching fix.",
"response": "Pages now load from cache on repeat visits. Nothing to do on your end."}
 

Fig. 04/Two example rows/File extension is .jsonl. instruction and response are required, context is not.

The most common mistake

Seen in almost every first run

Training loss goes down, so it looks like it is working. The model has learned nothing.

check for this first if results look wrong
check before training print(ds["text"][0])
<start_of_turn>user
Summarise this bug report for the standup.<end_of_turn>
<start_of_turn>model
Safari 17 drops our session cookie...<end_of_turn> ← correct
 
instruction: Summarise this bug report... ← template was skipped

Print the first row and read it before you train.

● Where to get rows

Most of your dataset already exists in things you have written.

01Sent emailYour reply is the response. The message you replied to is the instruction
02Code reviewThe diff is the context. Your review comment is the response
03Support ticketsUsually the best source, and usually unused
04Your own writingDocs, posts, commit messages. Best source for matching tone
05Chat historyPlaces where you corrected a model. Your correction is the response you wanted
06Written nowFifty rows you write tonight are worth more than five thousand you scraped

Collect examples that already exist before you write anything new.

● Your turn

Write one row now, by hand.

Fig. 05/Blank slip/Four minutes. Paper, notes app, anywhere.

Training rowNo. 000001
Instruction — what someone asks you
Response — how you would answer
Source — where this came from e.g. my sent mail, 14 Aug, to a customer
Write the answer you actually gave, not an improved version of it.
● Before you trust an archive

Measure how much of it you actually wrote.

A real check run on this laptop this afternoon. The git history looked like the best writing sample available, so we counted before using it.

5,384Commits written under my namelooks like a large personal corpus
3,955Carry a Co-Authored-By: Claude trailer73 percent. not my writing
1,431The rest, which read like the assistant tooolder sessions, and an auto-commit hook

Train on that and you get a model that writes like your tools while everyone believes it captured you. In 2026 a personal archive is contaminated by default, so this is a counting job you do first, not a worry you have later.

● How many rows do I need

1,000 rows

is what the test run used. For changing tone and format, 50 to 200 is usually enough. Start small and add more once you can measure the difference.

Fig. 06/Test run setup/First 1,000 lines of Dolly-15k. Gemma 4 E4B, 3 epochs, rank 8, 14 minutes of training.

● Coverage

Include every kind of answer you want, not fifty copies of the easy one.

×400The same simple case, repeatedafter about twenty examples, more adds nothing
×0The difficult case you care aboutnot covered, so the model will guess
×12Saying no, empty input, hostile usera small number is usually enough

The dictionary project had the same problem: volunteers sent thousands of examples for common words and none for the hard ones. The fix was asking for specific missing words, not asking for more volunteers.

● Ten minutes, before you write more rows

List the types of answer you want, and set a target for each.

Example

  • Short factual answer · 20 rows
  • Longer explanation for a beginner · 15 rows
  • Rewrite this text in our tone · 25 rows
  • Decline politely, with a reason · 10 rows
  • Input is unclear, ask a question back · 8 rows

Why bother

  • The model copies the balance of your data, not just the content
  • If 90 percent of rows are one type, most answers will look like that type
  • Counting takes ten minutes and prevents a wasted training run
  • The same list becomes the categories in your test set

Fig. 07/Coverage list/Write it before the file, not after.

● Using a big model to help

Let it write the questions. Do not let it write the answers.

Works well

  • Rewording instructions for answers you already have
  • Filling in a category where you only have two examples
  • Inventing awkward inputs you would not think of
  • Anything where the response is still written by you

Causes problems

  • Generating responses too. You end up copying the big model, not yourself
  • Everything starts to sound the same
  • Made-up facts get into the data and you train on them
  • Rows nobody read. If you have not read it, you do not know what is in it

Read every generated row before it goes in the file.

● Where each row came from

Add a source field to every row.

It costs nothing while you are writing and is impossible to reconstruct later. Three months from now, when a row looks wrong, this is the only way to find out where it came from.

01Tag itWhich file, thread, ticket or person the row came from
02Check rightsCustomer text, licensed data and model output each have different rules
03Remove personal dataNames, emails, API keys, account numbers. A fine-tuned model can repeat them back

Fig. 08/Source field/One of the dictionary's most prolific contributors sent thousands of entries from a psychiatric hospital. Good work, and the editors still needed to know where it came from.

● Twenty minutes

Check the file before it goes anywhere.

validate.py runs on your laptop · no GPU needed
$ python validate.py data/mine.jsonl
203 rows parsed
7 duplicate instructions → removed
2 rows with an empty response → removed
1 response over 2,048 tokens → shorten it or drop it
4 rows also in the test set → removed, see below
counts by type: answer 61 · explain 44 · rewrite 58 · decline 9 · clarify 7
189 rows written to data/mine.clean.jsonl
 

The last one matters most. If a row you trained on is also one of your test questions, the model has seen the answer and your comparison means nothing.

● A real run, start to finish

5,046 messages in. 513 rows out.

build_voice_dataset.py two months of one chat export
5,046 messages exported 134 conversations
2,028 written by me minus notes-to-self
758 are a reply to something merged bursts, dropped slow replies
 
160 bare acks over quota → capped, not deleted
58 duplicate responses → removed
24 touched a credential → whole exchange dropped
2 reply was only a link → removed
1 credential paste → removed
513 rows written 49 conversations, 23 held out
 

Fig. 09/Yield/Ten percent of the export survived, and that is a normal, healthy number. You are filtering for examples worth learning from.

Write your test questions tonight. On Friday it is already too late.

Questions that are not in your training file · kept in a separate file · the only way to judge Friday

● Five questions, five types

The test set from the example run.

prompts/eval_prompts.json never used for training
closed_qa  What is the capital of France, and why is it called the City of Light?
creative_writing  Write a four-line poem about a cat watching rain through a window.
general_qa  Give me three concrete tips for someone learning guitar in month one.
summarization  In two sentences, summarise what LoRA does and why it is useful.
classification  Classify this sentence and explain in one line: "The coffee was burnt
 but the conversation was wonderful."

Fig. 10/Test questions/Write five of your own, covering the types you listed on slide 14.

● The main block

Five steps. Everyone finishes with a file.

01Pick one jobOne thing you want the model to do. Write it as a single sentence
02List the typesFive to seven kinds of answer, with a target count for each
03Open your sourcesSent mail, tickets, review comments. Collect what exists first
04Write 50 rowsOne JSON object per line, instruction and response on each
05Hold back 5A separate file, never trained on. These are your test questions

Fifty rows plus five test questions is a complete dataset. Everything after tonight is more of the same.

● Friday, not tonight

What your file goes through next session.

32sLoadGemma 4 E4B in 4-bit. Uses about 10 GB of GPU memory
72sBeforeYour test questions, answered by the untrained model. Saved for comparison
862sTrainThe fine-tune itself. Fourteen minutes
20sAfterThe same questions, answered by your tuned model
—compare.mdBefore and after side by side, one question at a time

Fig. 11/Measured times/Nebius H100, 1 May 2026. All five answers changed. The trained adapter file was 30 MB. Sixteen minutes total, which is why tonight is the longer half.

● Why tonight is worth three hours

Four things that go wrong on Friday. Three of them start here.

01Loss is flat, or NaN, from the first stepthe chat template was not applied to the file · data
02Loss drops but answers are unchangedthe trained adapter is not loaded · not data, a training-day fix
03All answers changed, all of them worsetoo few rows, or too little variety · data
04One answer changes a lot, the rest not at alla training row is nearly identical to a test question · data

You cannot fix any of the three by changing a setting on Friday. They are fixed in the file, tonight.

● Before you go

What you are leaving with.

Yours to keep

  • A .jsonl dataset you wrote, with sources recorded
  • A separate test set, written before any training
  • A list of the answer types you covered, and the counts
  • A file that is ready to train on without changes

For Friday

  • Put the dataset in git tonight, while it is fresh
  • Bring both files and the laptop you wrote them on
  • Session 2 is Friday 22 August, 6 PM, same room and link

Fig. 12/Next session/Friday is the GPU night. Everything you built tonight is the input to it.