Goal
Estimate total FLOPs, PF-days, calendar time, and GPU-hours for a training run.
Training Compute Estimator (6ND Rule)
Estimate total FLOPs, PF-days, calendar time, and GPU-hours for a training run.
Key facts
- What it does
- Estimate total FLOPs, PF-days, calendar time, and GPU-hours for a training run.
- Formula
FLOPs ≈ 6 × params × tokens. Seconds = FLOPs / (gpus × TFLOPS×1e12 × util). PF-days = FLOPs / 8.64e19.- You enter
- Parameters · Training tokens · Per-GPU throughput · GPU count · Utilization (MFU)
- Worked example
- 0.69 PF-days; 1.45 days; about 278 GPU-hours.
A clearer path to an answer
From your question to a useful result
This page keeps the calculation transparent: define the goal, enter the matching values, inspect the method, and decide what the result means in your situation.
Inputs
Parameters · Training tokens · Per-GPU throughput · GPU count · Utilization (MFU)
Method
FLOPs ≈ 6 × params × tokens. Seconds = FLOPs / (gpus × TFLOPS×1e12 × util). PF-days = FLOPs / 8.64e19.
Next step
Calculate, review the assumptions below, then compare a related tool when the decision needs more context.
Training Compute Estimator (6ND Rule)
Estimate total FLOPs, PF-days, calendar time, and GPU-hours for a training run.
Open the Training Compute Estimator (6ND Rule) pageMore technology tools
Download PDFDownload Word (.doc)
Result
Enter your values above and choose Calculate to see the result here.
Calculation map
Follow the path from input to answer
01
Inputs (5)
- Parameters Ready
- Training tokens Ready
- Per-GPU throughput Ready
- GPU count Ready
- +1 more input
02
Formula
FLOPs ≈ 6 × params × tokens. Seconds = FLOPs / (gpus × TFLOPS×1e12 × util). PF-days = FLOPs / 8.64e19.
Bounded, transparent calculation
03
Result
- Calculate to preview the result.
Recent runs
Your recent runs stay in this browser session only.
Formula, assumptions, and example
Formula: FLOPs ≈ 6 × params × tokens. Seconds = FLOPs / (gpus × TFLOPS×1e12 × util). PF-days = FLOPs / 8.64e19.
The 6ND rule approximates transformer training compute. Dividing by achieved cluster throughput gives wall-clock time; multiplying by GPU count gives billed GPU-hours.
- Dense transformer training; MoE, sparsity, and recompute change the constant.
- Throughput and utilization are sustained averages you supply.
- One PF-day = 10^15 FLOPs sustained for a day.
Worked example: 0.69 PF-days; 1.45 days; about 278 GPU-hours.
Displayed input contract
- Parameters · minimum 0.001 · maximum 100000
- Training tokens · minimum 0.001 · maximum 100000000
- Per-GPU throughput · minimum 0.01 · maximum 1000000
- GPU count · minimum 1 · maximum 1000000
- Utilization (MFU) · minimum 0.01 · maximum 100
The displayed limits are checked before the handler runs. Model-specific domain checks may also reject impossible or non-finite inputs.
Methodology: This calculator follows the WorldCalculate input, formula, precision, and boundary policy. Read the official methodology.
Calculator usage statistics
Usage of this calculator and related tools
This section counts anonymous successful Calculate submissions, not unique visitors. Counts and top tools appear only when trusted aggregate data is available; country analysis is shown only under the same condition and reporting threshold.
Total successful calculations
Top tools
Country analysis
Most-used tools
Where usage is reported
Answer-first guide
How to use the Training Compute Estimator (6ND Rule) for a real question
Estimate total FLOPs, PF-days, calendar time, and GPU-hours for a training run. Start with one clearly defined goal, enter values in the units shown, and keep the result attached to the assumptions below.
What this answers
This tool is useful when your question includes training FLOPs, 6ND rule, GPU hours. It returns the outputs declared in the calculator contract rather than a live quote, approval, diagnosis, or professional sign-off.
What you enter
Parameters · Training tokens · Per-GPU throughput · GPU count · Utilization (MFU). Keep the same time period, unit system, and currency wherever the form requires comparable values.
How to check it
Run the worked example first, compare its output with the page's example, then change one input at a time. This makes an unexpected result easier to trace to a unit, boundary, or assumption.
Three checks before you rely on the answer
- Match the question. Confirm that the result means the quantity you need, not a similar-sounding percentage, balance, rate, or estimate.
- Match the inputs. Use the requested units and period, and read each hint before replacing the example values with your own.
- Read the boundary. Review the assumptions and limits. Dense transformer training; MoE, sparsity, and recompute change the constant.
Need a wider view? Browse Technology Calculators or compare the related tools below. The WorldCalculate methodology explains how formulas, examples, limits, and revisions are reviewed.
How to use the Training Compute Estimator (6ND Rule)
- Enter Parameters (billions).
- Enter Training tokens (billions).
- Enter Per-GPU throughput — Sustained dense throughput, not peak vendor numbers. (TFLOPS).
- Enter GPU count.
- Enter Utilization (MFU) (%).
- Choose Calculate and read the result panel.
- Use Download PDF or Download Word to save a result sheet.
Formula
FLOPs ≈ 6 × params × tokens. Seconds = FLOPs / (gpus × TFLOPS×1e12 × util). PF-days = FLOPs / 8.64e19.
The 6ND rule approximates transformer training compute. Dividing by achieved cluster throughput gives wall-clock time; multiplying by GPU count gives billed GPU-hours.
Worked example
0.69 PF-days; 1.45 days; about 278 GPU-hours.
Assumptions and limits
- Dense transformer training; MoE, sparsity, and recompute change the constant.
- Throughput and utilization are sustained averages you supply.
- One PF-day = 10^15 FLOPs sustained for a day.
Context and background
The representations behind technical results
Computing tools distinguish number bases, decimal and binary storage, network prefixes, transfer units, and visual ratios before performing the arithmetic.
Technical systems use several measurement conventions because software, hardware, networks, and design each describe quantities differently. Explicit representation prevents a familiar abbreviation from hiding a different definition.
Research and review
How this guide was researched
Researched by Hassan ALRowaie, Founder and editorial researcher at WorldCalculate.
This guide follows the live calculator's declared inputs, formula, worked example, assumptions, validation boundaries, and source-backed methodology. The review date describes editorial review of the calculator explanation; it is not a promise that external facts or rates remain current.
A training run can be described in several different ways: parameter count, token count, floating-point operations, elapsed time, or the number of accelerators occupied. Those descriptions are related, but they are not interchangeable. This estimator gives them a common arithmetic frame for a dense transformer workload. It starts with the approximate 6ND rule, where N is the parameter count and D is the number of training-token presentations, then converts the result into PF-days, calendar time, and GPU-hours using a supplied per-GPU throughput, GPU count, and model FLOPs utilization percentage. The result is a planning estimate rather than a benchmark, invoice, convergence forecast, or hardware recommendation. Its value comes from making the assumptions visible. If the model, token workload, measured throughput, and utilization interpretation are recorded beside the result, the estimate can support a useful comparison between training scenarios without pretending that a short formula captures every operation in a real distributed run.
What FLOPs measure
A FLOP is one floating-point operation, such as an addition or multiplication performed on floating-point values. A FLOP count is therefore a measure of arithmetic work, not a measure of memory capacity, electricity, elapsed time, model quality, or financial cost. Two workloads can require a similar number of FLOPs while behaving differently because one moves more data, waits on communication, or uses less efficient kernels. The count is still useful because it gives a hardware-independent way to describe the approximate amount of numerical work in a computation.
Training combines many operations. A forward pass produces predictions, a loss calculation compares them with the training target, and backpropagation computes gradients. An optimizer then updates parameters, while input preparation, checkpointing, evaluation, and communication add work around the main arithmetic. The 6ND estimate focuses on the dominant dense model arithmetic. It does not claim to enumerate every instruction executed by a training system, and it should not be read as a direct count of memory reads, network transfers, or all framework overhead.
FLOPs are especially useful for comparing a fixed workload across hardware or for comparing workloads on the same hardware. If the parameter count or token count doubles while the model family and approximation stay comparable, the estimated arithmetic work also doubles. Turning that work into time requires a throughput assumption. This calculator keeps those two questions separate: first estimate the work, then divide by an effective aggregate throughput that reflects the supplied GPU count and utilization.
- FLOPs describe approximate arithmetic work, not a promise of quality or speed.
- Memory traffic, communication, data loading, and checkpointing are not automatically included in a FLOP count.
- A useful estimate keeps workload size and achieved throughput as separate assumptions.
- The same FLOP total can take different times on different systems.
Why the 6ND approximation is used
Let N be the number of model parameters and D be the number of token presentations used for training. The dense approximation says that total training work is about 6 x N x D FLOPs. The rough derivation starts with a forward pass through dense weights. A multiply-accumulate is commonly counted as two operations, so the forward computation is approximately 2ND. Backpropagation through those weights requires roughly twice the forward arithmetic, or about 4ND. Adding forward and backward work gives about 6ND.
This derivation is a useful mental model rather than a universal identity. It assumes that the parameters represented by N participate densely for the tokens represented by D. It also treats the main matrix operations as the dominant cost and folds many architecture details into the approximation. Embedding operations, output projections, attention terms, normalization, activation functions, optimizer updates, and other operations can shift the actual total. The shift becomes more important when a model is unusually small, sequences are unusually long, or the architecture is not a conventional dense transformer.
The constant six also depends on what the author of a FLOP estimate counts as one operation. Some accounting conventions count a fused multiply-add as one operation and others count it as two. A comparison is meaningful only when the convention is consistent. This calculator follows the common 6ND convention represented by its formula. It does not attempt to replace a model-specific operation count, a profiler trace, or a measured throughput report.
- Forward arithmetic is roughly 2ND under the multiply-plus-add convention.
- Backward arithmetic is roughly 4ND in the same simplified view.
- The sum gives the dense training approximation 6ND.
- The factor six is a modeling constant, not a law that applies unchanged to every architecture.
Converting billions into FLOPs
The Parameters field and Training tokens field are both expressed in billions. If the entered values are paramsB and tokensB, then N equals paramsB x 10^9 and D equals tokensB x 10^9. Their product is paramsB x tokensB x 10^18. Substituting those conversions into 6ND gives FLOPs = 6 x paramsB x tokensB x 10^18. This billion-unit conversion is essential. Multiplying the two visible field values by six alone produces a compact coefficient, not a raw FLOP count.
The default values are 1 billion parameters and 10 billion training tokens. They produce N = 1 x 10^9 and D = 10 x 10^9, so the estimated work is 6 x 1 x 10 x 10^18 = 6 x 10^19 FLOPs. The value is large because a parameter participates in many operations across many token presentations. A token count is not merely the number of unique examples in a data folder; it is the number of token positions processed by the training computation under the chosen workload definition.
The fields accept decimal values, so a scenario such as 1.5 billion parameters or 2.5 billion tokens is valid when the source estimate is genuinely fractional in billions. The conversion still applies in exactly the same way. Keep the displayed units attached to the inputs when copying them into a note or spreadsheet. Entering a raw parameter count that is already multiplied by 10^9 into a field that expects billions would multiply the result by another billion and make the estimate meaningless.
- N = paramsB x 10^9 parameters.
- D = tokensB x 10^9 token presentations.
- FLOPs = 6 x paramsB x tokensB x 10^18 under the catalog convention.
- The visible input values are billions, not raw counts.
PF-days and the hardware-independent work total
A PF-day is the amount of work performed by a sustained one-petaflop system for one day. One petaflop is 10^15 FLOPs per second, and one day contains 86,400 seconds, so one PF-day is 10^15 x 86,400 = 8.64 x 10^19 FLOPs. The estimator divides the 6ND result by 8.64 x 10^19 to produce PF-days. This is a normalized work quantity. It does not say that the run has one petaflop of hardware or that the run will finish in the displayed number of days.
Combining the unit conversion with the 6ND rule gives PF-days = (6 x paramsB x tokensB x 10^18) / (8.64 x 10^19). The coefficient is approximately 0.0694444, so a quick check is PF-days approximately 0.0694444 x paramsB x tokensB. For the default product of 10, the result is about 0.694444 PF-days. This value remains the same if the same work is spread across one GPU, eight GPUs, or a larger idealized cluster.
PF-days should be kept distinct from calendar days. Calendar time depends on the effective throughput actually delivered by the selected GPU count and workload. A large PF-day total can finish quickly on a large efficient cluster, while a smaller total can take longer on a small or poorly utilized system. PF-days are best used to compare work; calendar time is used to plan a schedule; GPU-hours are used to describe accelerator occupancy.
- One PF-day equals 8.64 x 10^19 FLOPs.
- PF-days normalize work and do not depend on the chosen GPU count.
- The default 1B-parameter, 10B-token workload is about 0.694444 PF-days.
- Calendar time requires a throughput assumption in addition to the work total.
Reading the input fields
Parameters is the dense model parameter count in billions. The accepted range begins at 0.001 and extends to 100,000 billions. Training tokens is the total token workload in billions, with a supported range from 0.001 to 100,000,000 billions. Those broad bounds make the calculator useful for small educational scenarios and very large planning scenarios, but they do not validate whether a particular model or dataset really has the entered size. A finite value inside the range can still describe the wrong workload if its unit or scope is misunderstood.
Per-GPU throughput is entered in TFLOPS and is bounded from 0.01 through 1,000,000. In this record, TFLOPS means the reference dense throughput for one GPU, stated as trillions of FLOPs per second. GPU count is an integer from 1 through 1,000,000. A fractional GPU count is rejected because the formula models a count of occupied devices, not a continuously divisible resource. Utilization is entered as a percentage from 0.01% through 100% and is converted internally to a fraction.
Before entering values, identify the scope of each one. Parameter count should refer to the model being trained, token count should refer to the training work being estimated, and throughput should refer to the same model family, numerical precision, batch regime, and parallel configuration as the planned run. A throughput figure from a different workload can be numerically plausible but still produce a misleading time. Record the provenance and measurement conditions even though the calculator accepts only the numeric value.
- Parameters and tokens are entered in billions.
- TFLOPS is a per-GPU reference rate, and GPU count must be a whole number.
- Utilization is entered as a percent, not as a decimal fraction.
- The numeric bounds prevent unsupported inputs but do not prove that an input describes the intended run.
Utilization percent versus utilization fraction
The utilization field is a model FLOPs utilization percentage, often abbreviated MFU. The calculator converts it by dividing by 100. Thus 40 means a fraction of 0.40, 75 means 0.75, and 100 means 1.00. The effective aggregate throughput is gpus x tflops x 10^12 x (util/100). This conversion is part of the formula, not a display-only choice. A percent describes a portion out of one hundred, while the multiplier used in arithmetic must be between zero and one.
Entering 40 when the field expects a percent applies a 0.40 multiplier, which is the intended interpretation. Entering 0.40 applies 0.004 after the internal division, reducing the throughput by a factor of 100 compared with the likely intention. Conversely, entering 100 produces no utilization discount. The field's label and unit should therefore be copied with the number whenever a scenario is shared. A result that is off by roughly two orders of magnitude often indicates a percent-versus-fraction error.
MFU is not identical to every possible hardware utilization metric. It describes how much of a reference model-FLOP rate is being achieved by the model workload under the selected accounting convention. A monitoring system might report device occupancy, memory utilization, or kernel activity using a different definition. Use a value that matches the formula's role. If a measured throughput already represents the final end-to-end achieved rate, avoid discounting it a second time; either use that achieved rate as the effective reference with 100% or choose a pre-utilization reference and apply MFU once.
- 40% becomes 0.40 inside the throughput formula.
- Do not enter 0.40 if the field is labeled percent unless you intentionally mean 0.40%.
- MFU is a model-throughput concept, not automatically the same as device occupancy.
- Apply one utilization interpretation once; double discounting makes time and GPU-hours too large.
Sustained throughput is not peak throughput
Peak TFLOPS is a theoretical capability under a narrow arithmetic condition. It may assume a particular data type, instruction pattern, clock state, and full use of specialized hardware. A training run spends time on more than ideal matrix arithmetic. It may wait for memory, synchronize gradients, move activations, handle uneven batches, launch kernels, or pause for input and checkpoint work. A peak number can therefore make a calendar estimate look much shorter than an actual run.
The catalog describes tflops as sustained dense throughput rather than a peak vendor number. That wording asks for a rate grounded in the intended workload or a carefully chosen benchmark. The separate utilization input then represents the fraction of the reference rate that the planned run is expected to achieve. This makes the distinction explicit, but it also means the two inputs must not be derived from overlapping measurements without care. If the supplied TFLOPS already includes the workload's final MFU, use a utilization of 100% for that effective rate rather than applying the same loss again.
A sustained rate should be measured under conditions that resemble the planned training job. Matching only the precision is not enough if the benchmark uses a different sequence length, batch size, parallel layout, model size, or number of nodes. A small benchmark can have different communication and memory behavior from a large run. When no measurement is available, use a conservative scenario range rather than presenting a peak number as a fact.
- Peak TFLOPS is a hardware ceiling, not a normal end-to-end training rate.
- Sustained throughput should reflect the model and workload conditions being planned.
- Use the utilization field to apply one clear efficiency assumption.
- A measured achieved rate and a pre-utilization reference rate should not both receive the same discount.
Worked default calculation
The default scenario uses 1 billion parameters, 10 billion training tokens, 150 TFLOPS per GPU, 8 GPUs, and 40% MFU. First convert the workload: N = 1 x 10^9 and D = 10 x 10^9. The 6ND estimate is 6 x 10^9 x 10 x 10^9 = 6 x 10^19 FLOPs. Dividing by 8.64 x 10^19 FLOPs per PF-day gives 0.694444 PF-days before display rounding.
Next calculate the effective cluster throughput. The utilization fraction is 40/100 = 0.40. One GPU contributes 150 x 10^12 x 0.40 = 60 x 10^12 effective FLOPs per second. Eight GPUs therefore contribute 480 x 10^12, or 4.8 x 10^14 FLOPs per second. Dividing 6 x 10^19 by 4.8 x 10^14 gives 125,000 seconds. Dividing by 86,400 gives approximately 1.446759 calendar days.
Finally, GPU-hours multiply the elapsed seconds by the eight occupied GPUs and divide by 3,600. The result is 8 x 125,000 / 3,600 = 277.7778 GPU-hours. The interface reports values using result precision, so the example is shown as about 0.69 PF-days, 1.45 days, and 278 GPU-hours. The extra digits are useful for checking the arithmetic, but they do not imply that the underlying assumptions are known to that precision.
- Work: 6 x 10^19 FLOPs.
- Normalized work: 6 x 10^19 / 8.64 x 10^19 = 0.694444 PF-days.
- Effective cluster rate: 8 x 150 x 10^12 x 0.40 = 4.8 x 10^14 FLOPs per second.
- Time: 125,000 seconds, or about 1.446759 days and 277.7778 GPU-hours.
Scenario comparisons
Changing the token workload is a direct change to the estimated work. If the default model processes 20 billion tokens instead of 10 billion, the FLOP count and PF-days double. With the same eight GPUs, 150 TFLOPS reference rate, and 40% MFU, calendar time doubles from about 1.4468 days to 2.8935 days, and GPU-hours double from about 277.8 to 555.6. The change is not caused by a different device; it is caused by twice as many token presentations.
Changing the parameter count has the same linear effect under 6ND. Moving from 1 billion to 10 billion parameters while leaving the token count at 10 billion multiplies work by ten. The estimate becomes 6.944444 PF-days, about 14.4676 days at the default effective cluster throughput, and about 2,777.8 GPU-hours. This is a scaling comparison, not a claim that a ten-times-larger architecture will have identical throughput or memory behavior.
Changing GPU count affects calendar time under the ideal linear-scaling assumption. Sixteen GPUs at the default per-GPU rate and MFU provide twice the aggregate throughput of eight GPUs, so the predicted time is about 0.7234 days. The ideal GPU-hour total remains about 277.8 because half the time is spent on twice the devices. In a real distributed run, communication and imbalance can lower efficiency, so the sixteen-GPU result is an optimistic scaling baseline unless its MFU was measured at that scale.
- Doubling tokens doubles FLOPs, PF-days, calendar time, and ideal GPU-hours.
- Doubling parameters has the same proportional effect in the 6ND model.
- Doubling GPUs halves predicted time only if effective per-GPU performance stays constant.
- A larger GPU count does not reduce the normalized PF-day work.
Calendar time, PF-days, and GPU-hours
Calendar time answers how long the arithmetic portion of the run occupies the schedule. The estimator obtains seconds by dividing FLOPs by aggregate sustained throughput, then converts seconds to days using 86,400 seconds per day. PF-days answer how much normalized work exists before choosing a machine. GPU-hours answer how many GPU-hours are occupied if the selected devices sustain the calculated runtime. These units serve different planning questions and should be reported together rather than collapsed into one number.
Under ideal linear scaling, GPU-hours have an important relationship to the GPU count. Starting with seconds = FLOPs / (gpus x tflops x 10^12 x util/100), multiplying seconds by gpus cancels the explicit GPU count. The ideal GPU-hour estimate is therefore FLOPs divided by the effective per-GPU rate and by 3,600. Adding GPUs changes wall-clock time but not the ideal arithmetic occupancy. This cancellation stops being exact when scaling changes the sustained rate or when devices wait for one another.
The label GPU-hours billed is an arithmetic occupancy estimate, not a price quote. It does not know a rental rate, reservation, minimum charge, preemption policy, queue delay, or whether all devices are charged during setup and checkpoint intervals. Similarly, calendar time here is active calculated compute time. A real project may need additional elapsed time for queueing, image preparation, data staging, validation, evaluation, recovery, and human review. Keep those schedule and cost components outside this formula.
- PF-days describe normalized work.
- Calendar days describe active compute time under the entered aggregate rate.
- GPU-hours describe ideal device occupancy and are not a provider invoice.
- Queueing, setup, idle periods, and recovery need a separate schedule model.
Scaling and communication overhead
The seconds formula assumes that aggregate throughput is the number of GPUs multiplied by the same effective per-GPU throughput. That is a useful first-order model of perfect scaling. Distributed training must usually synchronize gradients or parameters, exchange activations, coordinate pipeline stages, and move data across links. Those transfers compete with computation and can create waits. As the device count grows, communication can become a larger portion of each step even if the arithmetic work is unchanged.
Parallel layout also matters. Data parallelism can require collective gradient reductions. Tensor or sequence partitioning adds activation and partial-result exchanges. Pipeline parallelism can introduce bubbles when stages are not balanced. Different devices or uneven batches can leave faster workers waiting for slower ones. Input pipelines, checkpoint writes, evaluation intervals, and failure recovery can lower the average rate further. None of these effects is derived from the integer GPU count alone.
The cleanest way to represent scaling overhead in this calculator is to supply throughput and MFU measured at the intended scale. If an eight-GPU benchmark achieves a different effective rate from a one-GPU benchmark, use the eight-GPU evidence for the eight-GPU scenario. Do not assume that multiplying a single-device peak rate by a large device count creates a sustained cluster rate. When measured data is unavailable, run optimistic, expected, and conservative utilization scenarios and treat the spread as planning uncertainty.
- The formula assumes linear aggregate scaling unless the supplied rate captures a lower efficiency.
- Gradient exchange, activation transfers, pipeline bubbles, and imbalance can reduce scaling.
- Use a throughput or MFU measured at the planned GPU and node count when possible.
- Scenario ranges are safer than treating perfect scaling as a guarantee.
Dense, MoE, sparse, and recomputed workloads
The 6ND rule is most natural for dense training, where essentially the same model parameters participate in the computation for each token. A mixture-of-experts model has a total parameter inventory that may be much larger than the parameters activated for one token. Replacing total parameters with an active-expert count can make the arithmetic closer to token-level compute, but it may omit shared layers, routing work, capacity padding, load imbalance, and communication. Using total parameters may instead overstate the main expert matmul work while remaining relevant to other resource needs. This calculator has no expert-routing input, so a MoE result must be labeled as a deliberate approximation.
Sparsity is not automatically a FLOP reduction. If zero weights are skipped by structured kernels, the useful operation count and throughput may change. If zeros are merely present in a dense representation, the hardware may still execute the same dense operations. Unstructured sparsity can also reduce arithmetic while adding indexing or irregular memory costs. The right adjustment depends on the representation and the kernels used by the actual training stack, not only on a stated sparsity percentage.
Activation recomputation, also called checkpointing, saves memory by discarding some intermediate activations and calculating them again during backpropagation. That repeated forward work increases arithmetic and can increase runtime even when it allows a larger batch. The basic 6ND estimate does not have a recomputation factor. If recomputation is important, use a measured effective throughput or create a separate scenario with a documented adjustment rather than silently treating the dense baseline as exact.
- Dense 6ND assumes the represented parameters are active for each token.
- MoE total parameters and active parameters answer different questions.
- Sparsity saves work only when the representation and kernels exploit it.
- Recomputation adds work that the basic constant six does not explicitly model.
Data and model workload assumptions
D should represent the token presentations that the training computation actually processes. If a dataset is repeated for several epochs, the total token workload should include those repeated presentations when the estimate is intended to cover the full run. A unique corpus size and a total optimization workload are not necessarily the same. Packing, padding, filtering, and sequence construction can also change how many positions reach the model. Define the token-count convention before comparing two estimates.
N should describe the parameters participating in the dense training work under the chosen approximation. A parameter count in a model description may include frozen components, routing experts, tied embeddings, or other components whose contribution is different from an ordinary dense layer. Parameter-efficient fine-tuning is a common trap: counting only the newly trainable adapter parameters can understate the forward and backward cost of running the full base model. The calculator does not inspect a model graph, so the scope of N must be supplied by the user.
Sequence length, microbatch size, gradient accumulation, precision, and optimizer settings often change achieved throughput even if the total token count remains fixed. Gradient accumulation can preserve a target effective batch while changing synchronization frequency. Long sequences can make attention terms more significant. Evaluation and generation may consume additional compute but are not automatically included in D. State whether D includes only optimization tokens or also evaluation and auxiliary passes, then keep any excluded work as a separate schedule line.
- Include repeated token presentations when estimating a multi-epoch run.
- Do not confuse unique dataset tokens with total optimization tokens.
- Make clear whether N includes the full model, frozen components, or only a trainable subset.
- Batching and sequence choices affect throughput even when 6ND work is unchanged.
Choosing and validating a throughput value
A useful throughput measurement begins with the same model shape, precision, sequence structure, batch regime, parallel layout, and GPU type as the planned run. Warmup steps should be separated from steady-state measurements because compilation, memory allocation, and cache effects can distort a short sample. Count the tokens that the training loop actually processes during the measured interval, measure the active interval carefully, and note whether communication and input waits were included. A benchmark that excludes those waits is not an end-to-end sustained rate.
The tflops field is a per-GPU rate, while the time formula uses aggregate throughput. If a benchmark reports cluster throughput, divide it by the number of GPUs only when that produces a comparable per-GPU reference and document the conversion. If the benchmark reports an already achieved end-to-end rate after all efficiency effects, it can be used with a 100% utilization factor to avoid applying the same loss twice. If it reports a reference rate before the planned MFU loss, enter that reference and apply the measured or assumed percentage once.
Validate the estimate after a pilot run by comparing predicted and observed active compute time, processed tokens, and device occupancy. A mismatch does not automatically mean the formula is wrong. It can reveal that D excluded padding, that the run included evaluation or checkpoint time, that the throughput was measured at a different scale, or that the model is outside dense 6ND assumptions. Use the discrepancy to refine the inputs and record the reason for the revision rather than hiding it in an unexplained utilization change.
- Match benchmark conditions to the intended model and distributed configuration.
- Separate warmup and setup from steady-state training throughput.
- Know whether a reported rate is per GPU, cluster-wide, pre-utilization, or already achieved.
- Use pilot observations to revise assumptions and explain discrepancies.
Rounding and sensitivity
The calculator returns PF-days and calendar days with a limited display precision, and GPU-hours with a coarser display precision. The default underlying values are approximately 0.694444 PF-days, 1.446759 days, and 277.7778 GPU-hours, while the displayed example is about 0.69, 1.45, and 278. Rounding is appropriate for communication, but it should happen after the arithmetic. Rounding the inputs early can matter when scenarios are close or when a downstream calculation multiplies the result.
The model is linear in parameters and tokens. A one percent change in either input produces approximately a one percent change in FLOPs and PF-days, with the same proportional effect on time if throughput is held fixed. Time is inversely related to GPU count, reference TFLOPS, and the utilization fraction in this idealized formula. For example, changing utilization from 40% to 41% changes the time by a factor of 40/41, not by an absolute one-day subtraction. Small percentage changes can matter for a long run.
Sensitivity is best communicated with a range. If the parameter count is uncertain, calculate a low and high count. If MFU could plausibly be 35% to 50%, report both rather than selecting the most convenient value. Because throughput and utilization are in the denominator, an optimistic assumption can shorten the result disproportionately. A rounded number such as 1.45 days should therefore not be presented with a false sense of precision when the workload or sustained rate is known only approximately.
- Round displayed results after calculating from unrounded inputs.
- Parameters and tokens scale work linearly.
- Time decreases inversely with GPU count, TFLOPS, and utilization in the ideal model.
- Use low, expected, and high scenarios when inputs are uncertain.
A practical planning and validation workflow
Start by writing a workload definition. Identify the model version or architecture, the parameter-count convention, the total token presentations, the treatment of repeats and padding, and any work such as evaluation that is outside the estimate. This prevents a clean calculation from being attached to an ambiguous workload. Next identify the planned GPU count and whether the run is single-node or distributed. Device count without a parallel layout is not enough to predict scaling quality, but it is still needed for the ideal calendar calculation.
Enter a throughput reference that matches the intended numerical precision and batch regime, then enter MFU as a percentage. Run the default or expected case and record all five inputs with the three outputs. Recompute the FLOP total, PF-day conversion, effective throughput, seconds, and GPU-hour conversion independently. A short hand calculation is often enough to catch a missing billion conversion, a percent error, or a calendar-versus-GPU-hour mix-up.
Create at least three comparison cases: an optimistic case with strong sustained throughput, an expected case based on the best available measurement, and a conservative case with lower MFU or more workload. After a pilot, compare actual tokens per second and active elapsed time with the selected case. Update only the assumption that the evidence supports. This workflow turns the estimator into a traceable planning record rather than a single number detached from the run that it describes.
- Define N and D before choosing a throughput number.
- Record all inputs, units, parallel layout, and excluded work.
- Recompute the result independently before using it in a schedule.
- Compare optimistic, expected, and conservative cases with pilot evidence.
Common mistakes and useful checks
A billion-unit mistake is the most damaging arithmetic error. The visible values 1 and 10 mean one billion parameters and ten billion tokens, not one parameter and ten tokens. The formula therefore produces 6 x 10^19 FLOPs, not 60 FLOPs. Check the dimensions before checking the decimal result. If two estimates differ by a factor near 10^9, the first question should be whether one side used raw counts and the other used billions.
A utilization mistake is usually recognizable because the runtime differs by a factor near 100. The entered value 40 means 40%, which becomes 0.40. It does not mean 0.40 as a percentage. A peak throughput mistake usually produces an estimate that is too optimistic without an obvious arithmetic error. Ask whether the rate includes communication, memory waits, and the actual model. A GPU-hour mistake happens when someone reports calendar time as resource consumption or multiplies by the GPU count twice.
A good boundary check uses a small workload and a one-GPU scenario. Another check changes only tokens, parameters, GPU count, or utilization and confirms the expected proportional direction. If tokens double, PF-days and time should double. If GPUs double under ideal scaling, time should halve and GPU-hours should remain constant. If utilization rises, time and ideal GPU-hours should fall in inverse proportion. These checks test the structure of the estimate, not only one expected decimal.
- Verify that both workload fields are in billions before applying 6ND.
- Check that a percent was not entered as a fraction.
- Distinguish peak hardware capability from measured sustained throughput.
- Use proportional-change tests to catch unit and scaling errors.
What the estimator does not decide
This estimator does not decide whether a model will converge, how many tokens are needed for a target quality level, whether a dataset is valid, or whether an architecture is a good choice. It does not inspect loss curves, validation metrics, data duplication, tokenization quality, or the statistical relationship between the data and the objective. A lower FLOP estimate is not evidence of better training, and a larger estimate is not evidence of better results. Quality and scientific validity require separate measurements.
It also does not calculate memory capacity, activation memory, optimizer state, checkpoint storage, network bandwidth, power use, cooling, hardware availability, queue time, or rental price. A run can fit the arithmetic schedule and still fail because it runs out of memory, cannot move data quickly enough, or cannot reserve the requested devices. The GPU-hours output can be multiplied by a known rate in a separate budget, but this record contains no pricing or billing rules.
Finally, the estimator does not choose a parallel strategy, determine a minimum GPU count, model failures, or guarantee that all devices remain busy. It is not a job scheduler and it does not convert a nominal time into a delivery date. Treat its result as the active dense-compute portion of a scenario. Add system-specific resource checks, operational margin, validation runs, and non-compute schedule work before making a commitment.
- Not decided: convergence, accuracy, data quality, or tokenization quality.
- Not calculated: memory, storage, power, network capacity, pricing, or availability.
- Not selected: parallel strategy, failure policy, minimum GPU count, or delivery date.
- A training plan needs system measurements and operational margin beyond this arithmetic estimate.
How to report a result responsibly
A responsible report states the assumptions before the result. Include the parameter count and token count with their billion units, the per-GPU TFLOPS interpretation, the integer GPU count, and the MFU percentage. Then report estimated FLOPs, PF-days, active calendar time, and GPU-hours. Mention whether the throughput is measured at the intended scale, whether the workload is dense, and which work is excluded. This gives another reader enough context to reproduce the arithmetic instead of treating the result as a property of the model alone.
Use conditional language that matches the evidence. A clear statement is that the entered dense 6ND scenario represents a certain amount of approximate work and, at the entered effective aggregate throughput, requires a certain active compute time. That statement is precise about the model and cautious about reality. It does not turn a rounded estimate into a promise. If a range was calculated, report the range and explain which assumption creates it.
The estimator is most useful when it is revisited. Replace an initial throughput guess with a measured rate, separate optimization tokens from evaluation tokens, revise the utilization value when scaling evidence arrives, and keep the old scenario for comparison. The arithmetic remains simple, but the planning record becomes stronger as the inputs become more specific. A transparent approximation that is updated from evidence is more useful than a highly precise number whose units, workload, or measurement basis are unknown.
- Report inputs, units, model scope, and exclusions with every result.
- Label the result as an approximate dense-workload estimate.
- Separate active compute time from queueing, setup, evaluation, and recovery time.
- Update the scenario when measured throughput and scaling evidence become available.
Frequently asked questions
What is the Training Compute Estimator (6ND Rule)?
Estimate total FLOPs, PF-days, calendar time, and GPU-hours for a training run.
What is the formula for the Training Compute Estimator (6ND Rule)?
FLOPs ≈ 6 × params × tokens. Seconds = FLOPs / (gpus × TFLOPS×1e12 × util). PF-days = FLOPs / 8.64e19. The 6ND rule approximates transformer training compute. Dividing by achieved cluster throughput gives wall-clock time; multiplying by GPU count gives billed GPU-hours.
What do I need to use this calculator?
Enter Parameters, Training tokens, Per-GPU throughput, GPU count, Utilization (MFU), then choose Calculate.
What are the limits of this calculator?
Dense transformer training; MoE, sparsity, and recompute change the constant. Throughput and utilization are sustained averages you supply. One PF-day = 10^15 FLOPs sustained for a day.
Methodology
This calculator is part of the WorldCalculate library. Its formula, example, assumptions, input bounds, and output formatting follow the official methodology.
Use this calculator as part of a bigger plan
These WorldCalculate collections connect this tool with related questions while keeping each calculation separate and transparent.