Spaces:
Running
Running
File size: 3,658 Bytes
939262b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 |
==================== ENVIRONMENT INFORMATION ====================
transformers_version: 2.11.0
framework: PyTorch
use_torchscript: False
framework_version: 1.4.0
python_version: 3.6.10
system: Linux
cpu: x86_64
architecture: 64bit
date: 2020-06-29
time: 08:58:43.371351
fp16: False
use_multiprocessing: True
only_pretrain_model: False
cpu_ram_mb: 32088
use_gpu: True
num_gpus: 1
gpu: TITAN RTX
gpu_ram_mb: 24217
gpu_power_watts: 280.0
gpu_performance_state: 2
use_tpu: False
</pt>
<tf>bash
python examples/tensorflow/benchmarking/run_benchmark_tf.py --help
An instantiated benchmark object can then simply be run by calling benchmark.run().
results = benchmark.run()
print(results)
results = benchmark.run()
print(results)
==================== INFERENCE - SPEED - RESULT ====================
Model Name Batch Size Seq Length Time in s
google-bert/bert-base-uncased 8 8 0.005
google-bert/bert-base-uncased 8 32 0.008
google-bert/bert-base-uncased 8 128 0.022
google-bert/bert-base-uncased 8 512 0.105
==================== INFERENCE - MEMORY - RESULT ====================
Model Name Batch Size Seq Length Memory in MB
google-bert/bert-base-uncased 8 8 1330
google-bert/bert-base-uncased 8 32 1330
google-bert/bert-base-uncased 8 128 1330
google-bert/bert-base-uncased 8 512 1770
==================== ENVIRONMENT INFORMATION ====================
transformers_version: 2.11.0
framework: Tensorflow
use_xla: False
framework_version: 2.2.0
python_version: 3.6.10
system: Linux
cpu: x86_64
architecture: 64bit
date: 2020-06-29
time: 09:26:35.617317
fp16: False
use_multiprocessing: True
only_pretrain_model: False
cpu_ram_mb: 32088
use_gpu: True
num_gpus: 1
gpu: TITAN RTX
gpu_ram_mb: 24217
gpu_power_watts: 280.0
gpu_performance_state: 2
use_tpu: False
By default, the time and the required memory for inference are benchmarked. In the example output above the first
two sections show the result corresponding to inference time and inference memory. In addition, all relevant
information about the computing environment, e.g. the GPU type, the system, the library versions, etc are printed
out in the third section under ENVIRONMENT INFORMATION. This information can optionally be saved in a .csv file
when adding the argument save_to_csv=True to [PyTorchBenchmarkArguments] and
[TensorFlowBenchmarkArguments] respectively. In this case, every section is saved in a separate
.csv file. The path to each .csv file can optionally be defined via the argument data classes.
Instead of benchmarking pre-trained models via their model identifier, e.g. google-bert/bert-base-uncased, the user can
alternatively benchmark an arbitrary configuration of any available model class. In this case, a list of
configurations must be inserted with the benchmark args as follows.
from transformers import PyTorchBenchmark, PyTorchBenchmarkArguments, BertConfig
args = PyTorchBenchmarkArguments(
models=["bert-base", "bert-384-hid", "bert-6-lay"], batch_sizes=[8], sequence_lengths=[8, 32, 128, 512]
)
config_base = BertConfig()
config_384_hid = BertConfig(hidden_size=384)
config_6_lay = BertConfig(num_hidden_layers=6)
benchmark = PyTorchBenchmark(args, configs=[config_base, config_384_hid, config_6_lay])
benchmark.run()
==================== INFERENCE - SPEED - RESULT ==================== |