nanoGPT User Guide

nanoGPT is Andrej Karpathy's minimal, hackable reference implementation of GPT for training and finetuning small-to-medium language models. The entire model fits in a single ~330-line model.py, the training loop is one ~330-line train.py, and configuration is a "poor man's configurator" that runs Python config files and CLI overrides through exec() against module globals. The repo is not a Python package — there is no pip install nanogpt. The intended workflow is git clone, install the PyTorch ecosystem, then run scripts from the repo root.

This guide covers everything a user needs to train, sample, finetune, and benchmark with nanoGPT. The Shakespeare-character demo trains a fully working baby GPT in under ten minutes on a single GPU and remains the fastest way to verify your installation.

1. Quick Start

The fastest path from an empty directory to a trained, sampling baby GPT is three commands plus a clone. Run all of them from the repository root.

git clone https://github.com/karpathy/nanoGPT.git
cd nanoGPT
pip install torch numpy transformers datasets tiktoken wandb tqdm
python data/shakespeare_char/prepare.py
python train.py config/train_shakespeare_char.py
python sample.py --out_dir=out-shakespeare-char

Tip

If you are on a MacBook (no CUDA), append --device=cpu --compile=False --eval_iters=20 --log_interval=1 --block_size=64 --batch_size=12 --n_layer=4 --n_head=4 --n_embd=128 --max_iters=2000 --lr_decay_iters=2000 --dropout=0.0 to the train.py invocation. The character-level Shakespeare recipe is intentionally tiny so it runs end-to-end on a laptop CPU in a reasonable amount of time.

What just happened

Key features

• Single-file model in model.py (LayerNorm, CausalSelfAttention, MLP, Block, GPTConfig, GPT). Around 330 lines, fully readable in one sitting.
• Single-file training loop in train.py with cosine learning-rate decay, gradient accumulation, mixed-precision autocast, optional torch.compile, and DDP support out of the box.
• Loads HuggingFace GPT-2 weights (gpt2, gpt2-medium, gpt2-large, gpt2-xl) for finetuning or zero-shot evaluation via GPT.from_pretrained().
• Flash Attention path activates automatically on PyTorch 2.0+ (scaled_dot_product_attention).
• Optional Weights & Biases logging and bench.py micro-benchmarking script with torch.profiler integration.
• Five reference configs in config/: train GPT-2 124M, finetune Shakespeare, evaluate the four GPT-2 sizes, and the Shakespeare-char baby model.

2. Installation

nanoGPT is a clone-and-run repository. There is no PyPI package, no Docker image in the repo, and no installation script — you clone the source, install the PyTorch ecosystem, and run scripts directly.

System requirements

Important

The full GPT-2 124M reproduction requires roughly 1 node × 8 × A100 40GB for ~5 days (per the comment at the top of config/train_gpt2.py). The Shakespeare-char demo runs comfortably on a single consumer GPU — or a laptop CPU with the size overrides shown in Quick Start.

Step 1: Clone the repository

git clone https://github.com/karpathy/nanoGPT.git
cd nanoGPT

Step 2: Install dependencies

There is no requirements.txt shipped with this repo. The dependencies are the ones imported by the scripts:

pip install torch numpy transformers datasets tiktoken wandb tqdm

Note

If you only plan to run the Shakespeare demos and never touch OpenWebText, you can omit datasets. Conversely, if you never plan to load HuggingFace GPT-2 weights, you can omit transformers. The scripts only import what they need.

Step 3: Verify the install

python -c "import torch; print(torch.__version__, torch.cuda.is_available())"

2.5.1+cu121 True

If torch.cuda.is_available() returns False and you have an NVIDIA GPU, reinstall PyTorch with the CUDA wheel that matches your driver — see pytorch.org.

Step 4: Prepare a dataset

Pick one of the three bundled datasets:

Warning

data/openwebtext/prepare.py downloads roughly 54 GB into the HuggingFace cache and writes another ~17 GB of train.bin. It is single-shot but takes hours and a fast disk. Start with Shakespeare-char to confirm everything works.

Step 5: Run the Shakespeare-char demo

python train.py config/train_shakespeare_char.py

You should see output like:

tokens per iteration will be: 16,384
Initializing a new model from scratch
defaulting to vocab_size of GPT-2 to 50304 (50257 rounded up for efficiency)
number of parameters: 10.65M
num decayed parameter tensors: 26, with 10,727,424 parameters
num non-decayed parameter tensors: 13, with 4,992 parameters
using fused AdamW: True
compiling the model... (takes a ~minute)
step 0: train loss 4.2876, val loss 4.2823
iter 0: loss 4.2841, time 6829.40ms, mfu -100.00%
iter 10: loss 3.2447, time 23.65ms, mfu 5.46%
...
step 5000: train loss 0.6178, val loss 1.4974

When training finishes (~5,000 iters), a checkpoint will live at out-shakespeare-char/ckpt.pt.

3. Configuration

nanoGPT uses a deliberately bare-bones configuration mechanism nicknamed the "poor man's configurator." It lives in configurator.py and works like this:

1. train.py, sample.py, and bench.py declare default values as module-level globals.
2. They call exec(open('configurator.py').read()).
3. configurator.py walks sys.argv[1:]. If an arg has no =, it is treated as a Python config file and exec()'d into the calling module's globals. If an arg starts with --key=value, it overrides the matching global by parsing value with ast.literal_eval.
4. Any unknown key raises ValueError: Unknown config key: {key}.

The override pattern

# config file only
python train.py config/train_shakespeare_char.py

# config file + CLI overrides (overrides win)
python train.py config/train_shakespeare_char.py --batch_size=32 --device=cpu

# CLI overrides only, all defaults from train.py
python train.py --batch_size=32 --compile=False

Important

Order matters. The config file is exec()'d first, then --key=value overrides are applied after. CLI flags always win over config-file values, which always win over the train.py defaults.

Config file shape

A config file is just plain Python. It assigns values to the same names that train.py/sample.py/bench.py declare as defaults. Example (config/train_shakespeare_char.py):

out_dir = 'out-shakespeare-char'
eval_interval = 250
eval_iters = 200
log_interval = 10

always_save_checkpoint = False
wandb_log = False
wandb_project = 'shakespeare-char'
wandb_run_name = 'mini-gpt'

dataset = 'shakespeare_char'
gradient_accumulation_steps = 1
batch_size = 64
block_size = 256

n_layer = 6
n_head = 6
n_embd = 384
dropout = 0.2

learning_rate = 1e-3
max_iters = 5000
lr_decay_iters = 5000
min_lr = 1e-4
beta2 = 0.99
warmup_iters = 100

Most-used training settings

The full reference is in the Admin & Developer Guide. The settings below are the ones users tune most often.

Most-used sampling settings

Tip

The sampling defaults are deliberately chatty (num_samples=10, max_new_tokens=500). Knock those down with --num_samples=1 --max_new_tokens=100 for quick sanity checks.

4. Core Usage

The three scripts you will run day-to-day are train.py, sample.py, and bench.py. They share the configurator, so everything below works with both config-file syntax and --key=value overrides.

Training (train.py)

train.py is a single while-loop that:

1. Resolves config (defaults → config file → CLI overrides).
2. Detects DDP from the RANK environment variable. If unset, runs single process.
3. Builds an np.memmap data loader over data/{dataset}/train.bin and val.bin.
4. Instantiates the model from one of three init paths: 'scratch', 'resume', or 'gpt2*'.
5. Configures AdamW via model.configure_optimizers (2D tensors get weight decay, 1D tensors don't).
6. Optionally torch.compiles the model.
7. Wraps in DDP if RANK >= 0.
8. Runs the training loop with cosine LR scheduling, gradient accumulation, bfloat16/float16 autocast, and optional GradScaler for float16.
9. Every eval_interval iterations, evaluates train and val loss over eval_iters batches and writes ckpt.pt if val improved (or always, if always_save_checkpoint=True).

Single-GPU training

python train.py config/train_shakespeare_char.py
python train.py --batch_size=32 --compile=False

Distributed training (DDP) — single node

torchrun --standalone --nproc_per_node=4 train.py config/train_gpt2.py

torchrun sets RANK, LOCAL_RANK, and WORLD_SIZE for each process. train.py reads them, assigns each rank to cuda:{LOCAL_RANK}, picks the rank-0 process as master_process (handles logging, eval, and checkpointing), and divides gradient_accumulation_steps by WORLD_SIZE.

Distributed training — multi-node

# master node (123.456.123.456)
torchrun --nproc_per_node=8 --nnodes=2 --node_rank=0 \
  --master_addr=123.456.123.456 --master_port=1234 train.py

# worker node
torchrun --nproc_per_node=8 --nnodes=2 --node_rank=1 \
  --master_addr=123.456.123.456 --master_port=1234 train.py

Note

If your cluster lacks Infiniband, prepend NCCL_IB_DISABLE=1 to the torchrun command on every node. This is documented in the docstring at the top of train.py.

Resuming an interrupted run

python train.py config/train_gpt2.py --init_from=resume

init_from='resume' loads out_dir/ckpt.pt, restores iter_num and best_val_loss, and forces n_layer, n_head, n_embd, block_size, bias, vocab_size to match the checkpoint (other args like dropout can still be tuned).

Important

Keep out_dir stable when resuming. The script reads {out_dir}/ckpt.pt directly. If you change out_dir, copy the checkpoint over first.

Sampling (sample.py)

# from a checkpoint you trained
python sample.py --out_dir=out-shakespeare-char

# from a HuggingFace GPT-2 checkpoint
python sample.py --init_from=gpt2-xl

# with a prompt
python sample.py --out_dir=out-shakespeare-char --start="ROMEO:"

# from a prompt file
python sample.py --out_dir=out --start="FILE:prompt.txt" --num_samples=3

If the checkpoint's training config used a dataset with a meta.pkl (character-level), sample.py loads data/{dataset}/meta.pkl and uses its stoi/itos mapping. Otherwise it falls back to the GPT-2 BPE encoder via tiktoken.get_encoding("gpt2").

Benchmarking (bench.py)

bench.py is a stripped-down train.py that runs a fixed number of forward, backward, and optimizer steps and reports time-per-iteration plus model FLOPs utilization (MFU).

python bench.py
python bench.py --compile=False --real_data=False --profile=True

Mixed precision — bfloat16 vs float16

train.py autodetects: if the GPU supports bfloat16 (torch.cuda.is_bf16_supported()), it picks 'bfloat16'; otherwise 'float16'. The difference matters:

• bfloat16: same exponent range as float32, no GradScaler needed. The cleaner default on Ampere (A100, A6000), Hopper (H100), and Ada (RTX 4090).
• float16: narrower exponent range, requires torch.cuda.amp.GradScaler, which train.py enables automatically when dtype == 'float16'.

Warning

If you force --dtype=bfloat16 on a card that doesn't support it (anything pre-Ampere), training will silently fall back to slow paths or NaN out. Either let the autodetect run or explicitly set --dtype=float16.

Checkpoint contents

A checkpoint is a Python dict pickled by torch.save:

{
    'model':         state_dict,    # raw_model.state_dict()
    'optimizer':     state_dict,    # optimizer.state_dict()
    'model_args':    dict,          # n_layer, n_head, n_embd, block_size, bias, vocab_size, dropout
    'iter_num':      int,           # last completed iter
    'best_val_loss': float,         # for early stopping decisions
    'config':        dict,          # all of train.py's config_keys snapshot
}

Checkpoints land at {out_dir}/ckpt.pt. There is one checkpoint per out_dir — the file is overwritten each save.

5. Common Tasks

Task 1: Train the Shakespeare-char baby GPT in under 10 minutes

python data/shakespeare_char/prepare.py
python train.py config/train_shakespeare_char.py
python sample.py --out_dir=out-shakespeare-char --start="JULIET:"

This trains a 6-layer / 6-head / 384-embed model (~10.65M params) for 5,000 iterations at batch_size=64, block_size=256, character-level vocab (65 unique chars). Final val loss lands around 1.5, which produces recognizable mock-Shakespeare.

Task 2: Finetune a public GPT-2 checkpoint on Shakespeare

python data/shakespeare/prepare.py
python train.py config/finetune_shakespeare.py
python sample.py --out_dir=out-shakespeare --start="To be or not to be,"

config/finetune_shakespeare.py sets init_from='gpt2-xl', learning_rate=3e-5, decay_lr=False, max_iters=20, gradient_accumulation_steps=32, always_save_checkpoint=False. It only saves when validation improves and finetunes at constant LR. Swap init_from for 'gpt2', 'gpt2-medium', or 'gpt2-large' if gpt2-xl is too big for your GPU.

Tip

For finetuning, bump dropout from 0.0 to 0.1 or so. The default train.py config sets it to 0.0 because that's optimal for pretraining, but a small amount of regularization helps when training on tiny corpora.

Task 3: Sample from a HuggingFace GPT-2 checkpoint with no training

python sample.py \
  --init_from=gpt2-xl \
  --start="What is the answer to life, the universe, and everything?" \
  --num_samples=5 --max_new_tokens=100

sample.py notices init_from.startswith('gpt2'), calls GPT.from_pretrained('gpt2-xl', dict(dropout=0.0)), and that helper pulls weights from HuggingFace via transformers.GPT2LMHeadModel.from_pretrained, then maps them onto nanoGPT's vanilla nn.Linear layout (transposing the GPT-2 Conv1D weights).

Task 4: Pretrain GPT-2 124M on OpenWebText across 8 GPUs

python data/openwebtext/prepare.py
torchrun --standalone --nproc_per_node=8 train.py config/train_gpt2.py

config/train_gpt2.py is the reference script for "GPT-2 124M down to a very nice loss of ~2.85 on 1 node of 8X A100 40GB" (≈5 days). It sets batch_size=12, block_size=1024, gradient_accumulation_steps=40 (5 × 8 GPUs), max_iters=600000, lr_decay_iters=600000, weight_decay=1e-1, with wandb logging on under project 'owt'.

Task 5: Log to Weights & Biases

pip install wandb
wandb login
python train.py config/train_gpt2.py --wandb_log=True \
  --wandb_project=my-experiments --wandb_run_name=run-001

When wandb_log=True and the process is master_process, train.py calls wandb.init and logs iter, train/loss, val/loss, lr, and mfu at each eval_interval.

Caution

Never commit your wandb API key to source control. wandb login writes it to ~/.netrc. If you must run on a shared box, use wandb login --relogin in your shell, not in a config file.

Task 6: Scale the model to a custom dataset

mkdir -p data/mydata
# write your own data/mydata/prepare.py that produces train.bin / val.bin

python data/mydata/prepare.py

python train.py \
  --dataset=mydata \
  --out_dir=out-mydata \
  --n_layer=8 --n_head=8 --n_embd=512 \
  --block_size=512 \
  --batch_size=32 --gradient_accumulation_steps=4 \
  --learning_rate=3e-4 --min_lr=3e-5 \
  --max_iters=20000 --lr_decay_iters=20000 \
  --warmup_iters=200 \
  --eval_interval=500 --eval_iters=100

train.py reads data/{dataset}/train.bin and val.bin directly. If you also drop a meta.pkl with vocab_size, the training run picks it up automatically; otherwise nanoGPT defaults to vocab_size=50304 (the GPT-2 vocabulary padded up from 50,257 to the nearest multiple of 64 for kernel efficiency).

Task 7: Prepare a custom dataset

The Shakespeare-char prepare.py is the cleanest template. The contract train.py expects is two NumPy uint16 files:

import numpy as np

# encode your text however you want - char-level, BPE, sentencepiece, etc.
train_ids = ...  # list[int]
val_ids   = ...  # list[int]

train_ids = np.array(train_ids, dtype=np.uint16)
val_ids   = np.array(val_ids,   dtype=np.uint16)
train_ids.tofile('data/mydata/train.bin')
val_ids.tofile('data/mydata/val.bin')

# optional but recommended for char-level vocabs
import pickle
with open('data/mydata/meta.pkl', 'wb') as f:
    pickle.dump({'vocab_size': vocab_size, 'stoi': stoi, 'itos': itos}, f)

Note

np.uint16 caps your vocab at 65,536. The GPT-2 BPE vocab (50,257) and common BPE sizes fit comfortably. If you need a vocab >65K, switch to np.uint32 and update the dtype= arguments in train.py's get_batch and bench.py's get_batch to match.

Task 8: Recover or resume an interrupted run

If your training crashed or you Ctrl-C'd it:

python train.py config/train_gpt2.py --init_from=resume

init_from='resume' reads {out_dir}/ckpt.pt, restores iter_num and best_val_loss, and continues. The optimizer state is also restored, so the LR schedule picks up exactly where it left off.

Warning

If you change n_layer, n_head, n_embd, block_size, bias, or vocab_size between runs, --init_from=resume will overwrite your CLI overrides with the checkpoint's values to keep the architecture compatible. Other knobs like dropout or learning_rate can still be tuned freely.

6. Troubleshooting

Error: RuntimeError: Dataset scripts are no longer supported

Cause: As of recent HuggingFace datasets releases, the openwebtext dataset stopped publishing as a scripted dataset. This is tracked upstream in GitHub issue #633.

Solution:

1. Pin an older datasets version: pip install "datasets<2.20".
2. Or use a community mirror that still ships the parquet shards directly.
3. Or replace data/openwebtext/prepare.py with a hand-rolled tokenizer over any large web-text corpus you can supply yourself — the only contract train.py cares about is train.bin / val.bin as uint16 token files.

Prevention: This is a moving target on the HuggingFace side. If you must reproduce GPT-2 124M end-to-end, lock your datasets version in a venv.

Error: ValueError: Unknown config key: {key}

Cause: You passed --{key}=value on the command line (or set it in a config file) but {key} is not a global declared in train.py/sample.py/ bench.py. The configurator deliberately rejects typos so a misspelled flag doesn't silently no-op.

Solution: Double-check the spelling against the tables in Section 3 of this guide. Common gotchas: lr (the right name is learning_rate), epochs (nanoGPT uses max_iters, not epochs), dropout_rate (the right name is dropout).

Prevention: When in doubt, scan the # default config values block at the top of train.py (or sample.py / bench.py) — the configurator only recognizes names declared there.

Error: WARNING: using slow attention. Flash Attention requires PyTorch >= 2.0

Cause: model.py:CausalSelfAttention.__init__ checks hasattr(torch.nn.functional, 'scaled_dot_product_attention'). On PyTorch < 2.0 that attribute is missing, so the model falls back to a manual QK^T softmax with a registered triangular causal mask buffer.

Solution: Upgrade to PyTorch 2.0 or newer:

pip install --upgrade torch

Prevention: PyTorch 2.0 has been stable for a while; pin torch>=2.0 in any requirements file you maintain downstream.

Error: Training NaNs out within a few iterations

Cause: Common in float16 autocast when gradients overflow before GradScaler adapts; or when the learning rate is too high; or when gradient_accumulation_steps is set so high that loss scaling drifts. Issue #524 (Pretraining Divergence) is the long-running upstream thread for discussion of nanoGPT NaN modes.

Solution:

1. Switch to --dtype=bfloat16 if your GPU supports it (Ampere+).
2. Reduce learning_rate (try halving it).
3. Lower gradient_accumulation_steps until the issue stops, then bisect.
4. Verify grad_clip=1.0 is in effect (the default).

Prevention: On Ampere/Hopper/Ada GPUs, prefer bfloat16. The Karpathy default already does this when bfloat16 is supported.

Error: CUDA out of memory during training

Cause: Your model + activations + optimizer state + gradients exceed GPU memory.

Solution:

1. Lower batch_size and raise gradient_accumulation_steps proportionally to keep effective batch size constant.
2. Lower block_size (context length is quadratic in attention memory).
3. Drop compile=True — torch.compile increases peak memory during compilation.
4. Switch to a smaller model: lower n_layer, n_embd, or both.

Prevention: Use bench.py --profile=True on your target hardware before launching the long run — the resulting traces under ./bench_log show peak GPU allocation per step.

Error: Sampling outputs are gibberish even though training loss is low

Cause 1: Mismatched encoder. If you trained with character-level encoding (shakespeare_char / a custom meta.pkl) but sample.py defaulted to the GPT-2 BPE encoder, every "char" gets re-encoded as several BPE tokens of unrelated semantics.

Cause 2: You passed --init_from=gpt2* to sample.py but pointed --out_dir at a checkpoint trained from scratch. init_from='gpt2*' ignores out_dir.

Solution:

1. For Cause 1: confirm data/{dataset}/meta.pkl exists and that sample.py's Loading meta from ... print appears at startup.
2. For Cause 2: pick exactly one of --init_from=resume --out_dir=... or --init_from=gpt2-xl. They're mutually exclusive.

Prevention: Always sanity-check the first two lines of sample.py output — they tell you exactly which encoder and which weights are being used.

Error: connection refused / DDP processes hang on multi-node setup

Cause 1: master_addr unreachable from worker nodes (firewall, wrong IP).

Cause 2: Cluster lacks Infiniband and NCCL is trying to use it.

Solution:

1. Verify worker → master TCP reachability on master_port.
2. Prepend NCCL_IB_DISABLE=1 to every torchrun invocation: NCCL_IB_DISABLE=1 torchrun --nproc_per_node=8 ...
3. For deeper diagnostics, set NCCL_DEBUG=INFO.

Prevention: Test single-node DDP first (torchrun --standalone --nproc_per_node=2 train.py) before scaling out.

Diagnostic decision tree

1. Did the install verify? python -c "import torch; print(torch.__version__, torch.cuda.is_available())" — if not, fix PyTorch first.
2. Does python data/shakespeare_char/prepare.py produce train.bin? If not, it's a network or write-permission issue.
3. Does python train.py config/train_shakespeare_char.py make it past step 0? If not, see the OOM and NaN sections above.
4. Does python sample.py --out_dir=out-shakespeare-char produce readable output? If not, see the gibberish-sampling section.

If all four pass, your installation is healthy and any failures from there are configuration- or data-specific.

7. FAQ

Q: Is nanoGPT a Python package I can pip install?

No. nanoGPT is a clone-and-run repository. There is no setup.py, no pyproject.toml, no PyPI release. You git clone, pip install the PyTorch ecosystem manually, and run python train.py from the repo root.

Q: What is the smallest GPU I can train GPT-2 124M on?

config/train_gpt2.py was tuned for 8 × A100 40GB. To train 124M on a single GPU you'll need to keep the effective batch size by raising gradient_accumulation_steps and lowering batch_size. Even with that, expect dozens of days of compute. The Shakespeare-char demo is the realistic single-GPU and single-CPU target for hobbyists.

Q: Why vocab_size=50304 instead of the actual GPT-2 size of 50257?

train.py rounds 50257 up to the nearest multiple of 64 (= 50304) because "50304 % 64 == 0" lets the GPU run faster matmul kernels. The model wastes 47 unused embedding rows but in exchange every Linear layer hits an efficient tiling. The from_pretrained path forces vocab_size=50257 because the OpenAI GPT-2 checkpoints were trained with that size — but the default scratch path bumps it back up to 50304.

Q: How do I disable torch.compile?

Pass --compile=False on the CLI, or set compile = False in your config file. torch.compile requires PyTorch 2.0 and adds a ~1-minute warmup. On small models or quick experiments, it's often slower overall.

Q: Why does the validation loss differ slightly between runs even with fixed seed?

torch.manual_seed(1337 + seed_offset) is called once at startup, but non-determinism creeps in via cuDNN, async CUDA streams, and torch.compile fusions. For bit-exact reproducibility, also set torch.use_deterministic_algorithms(True) — but expect a slowdown.

Q: Where do I look for the model architecture?

Everything lives in model.py (~330 lines):

• LayerNorm — line 18
• CausalSelfAttention — line 29
• MLP — line 78
• Block — line 94
• GPTConfig (dataclass) — line 109
• GPT — line 118 (with from_pretrained, configure_optimizers, estimate_mfu, crop_block_size, generate)

Q: Can I train on Apple Silicon?

Yes for the small demos. Set --device=mps --dtype=float16 --compile=False on the train.py and sample.py commands. torch.compile on MPS is unstable, and bfloat16 on MPS isn't supported on all releases.

Q: How do I make sampling deterministic?

sample.py already calls torch.manual_seed(seed) and torch.cuda.manual_seed(seed) with seed=1337 by default. Override with --seed=42 if you want a fresh stream. For top-p (nucleus) sampling, pass --top_k=None and adjust --temperature instead — there is no native top-p flag.

Q: Where are the GPT-2 weights downloaded to?

GPT.from_pretrained calls transformers.GPT2LMHeadModel.from_pretrained which uses HuggingFace's standard cache (~/.cache/huggingface/hub/ by default, overridable via HF_HOME).

Q: Is there an HTTP API or a CLI binary?

No. nanoGPT is intentionally a script-only repo. There are no exposed endpoints, no argparse CLI, no installable entry points. Every interaction is python train.py … or python sample.py … from the repo root.

Q: How do I cite nanoGPT?

Per the LICENSE in the repo (MIT), you don't have to. If you want to, referencing the GitHub URL and commit hash is the standard practice for research code without a published paper.