mako datagen
Added in version 1.4.0
The datagen command allows you to generate synthetic datasets to be used with other mako commands.
The generated dataset is produced in a json format conformant to the dataset format explained in the dedicated guide.
Please follow the guide for information on how to integrate the resulting dataset into your workflow.
Command Syntax
Required arguments
- output_file (str, positional) - Filename to where to save the resulting dataset. The dataset will be saved as
json. - --dataset (str) - Dataset type to generate. Currently supported datasets are:
burstgpt,randomandprefix_repetition.
Common flags
These flags allow you to tweak the common behavior for all types of datasets.
--ssed, -s(int) - Randomness seed. Default: 42.-
--tokenizer, -t(str) - HF tokenizer name or path. The tokenizer is needed for converting tokens to text and vice-versa. Default: gpt2Note
Different tokenizers might tokenize the same text into a different number of resulting tokens. Since many of the parameters below deal with the number of tokens, but the dataset content is saved as text (to make it possible to use with different models), it is possible that the actual number of input or output tokens observed by your model during benchmarking will be different than the requested generation parameter. This is not a bug but rather a fundamental limitation.
In order to minimize any such discrepancy, if you know in advance what model will use the resulting dataset, it is recommenced to specify the right tokenizer for your model using this parameter. However, note that: 1) the dataset should be useable even if a different tokenizer is used for generation; and 2) even if the same tokenizer is used, some discrepancies can still happen, although should be relatively less severe.
-
--pretty, -p- Save the resulting json in a more human-friendly manner.
Burst-GPT dataset parameters
--dataset-path(str) - Path to the BurstGPT CSV file. The dataset will be downloaded if a given file does not exist. Default: datasets/burstgpt.csv.--force-download- For re-downloading the BurstGPT dataset file, even if it already exists.--model-filter- Filter the BurstGPT dataset by a model name. Default: GPT-4.--min-response-tokens- Filter the BurstGPT dataset by a minimum response length. Default: 1.
Random dataset parameters
--random-prefix-len(int) - Length of the common random prefix that will be appended to each request's input prompt. Default: 0.--random-range-ratio(float, 0-1) - Defines the maximum deviation from the provided input and output lengths. In particular, each request's input and output tokens will be in range[(1 - r) * x, (1 + r) * x], whereris the ratio parameter andxis the requested value. Default: 0.--random-input-len(int) - Number of input tokens to generate for each request. Also see--random-range-ratio. Default: 1024.--random-output-len(int) - Number of output tokens to generate for each request. Also see--random-range-ratio. Default: 128.
Prefix repetition dataset parameters
--prefix-repetition-prefix-len(int) - Number of tokens in each repeated prefix. Default: 256.--prefix-repetition-suffix-len(int) - Number of tokens appended to each prefix to form full input prompt. Default: 256.--prefix-repetition-num-prefixes(int) - Number of unique repeated prefixes, each with length--prefix-repetition-prefix-len. Default: 10.--random-output-len(int) - Number of output tokens to generate for each request. Default: 128.
Examples
Generate a dataset file with 10 requests following the data from the BurstGPT dataset, saving the result to result.json:
Generate random dataset with 10 requests, each having 110-150 input (30 come from the prefix) and 80-120 output tokens. The first 30 input tokens should be the same for all requests.
The tokenizer used when generating should be the tokenizer for Llama-3-2 instruct model.