Spaces:
Running
Running
| ==================== ENVIRONMENT INFORMATION ==================== | |
| transformers_version: 2.11.0 | |
| framework: PyTorch | |
| use_torchscript: False | |
| framework_version: 1.4.0 | |
| python_version: 3.6.10 | |
| system: Linux | |
| cpu: x86_64 | |
| architecture: 64bit | |
| date: 2020-06-29 | |
| time: 08:58:43.371351 | |
| fp16: False | |
| use_multiprocessing: True | |
| only_pretrain_model: False | |
| cpu_ram_mb: 32088 | |
| use_gpu: True | |
| num_gpus: 1 | |
| gpu: TITAN RTX | |
| gpu_ram_mb: 24217 | |
| gpu_power_watts: 280.0 | |
| gpu_performance_state: 2 | |
| use_tpu: False | |
| </pt> | |
| <tf>bash | |
| python examples/tensorflow/benchmarking/run_benchmark_tf.py --help | |
| An instantiated benchmark object can then simply be run by calling benchmark.run(). | |
| results = benchmark.run() | |
| print(results) | |
| results = benchmark.run() | |
| print(results) | |
| ==================== INFERENCE - SPEED - RESULT ==================== | |
| Model Name Batch Size Seq Length Time in s | |
| google-bert/bert-base-uncased 8 8 0.005 | |
| google-bert/bert-base-uncased 8 32 0.008 | |
| google-bert/bert-base-uncased 8 128 0.022 | |
| google-bert/bert-base-uncased 8 512 0.105 | |
| ==================== INFERENCE - MEMORY - RESULT ==================== | |
| Model Name Batch Size Seq Length Memory in MB | |
| google-bert/bert-base-uncased 8 8 1330 | |
| google-bert/bert-base-uncased 8 32 1330 | |
| google-bert/bert-base-uncased 8 128 1330 | |
| google-bert/bert-base-uncased 8 512 1770 | |
| ==================== ENVIRONMENT INFORMATION ==================== | |
| transformers_version: 2.11.0 | |
| framework: Tensorflow | |
| use_xla: False | |
| framework_version: 2.2.0 | |
| python_version: 3.6.10 | |
| system: Linux | |
| cpu: x86_64 | |
| architecture: 64bit | |
| date: 2020-06-29 | |
| time: 09:26:35.617317 | |
| fp16: False | |
| use_multiprocessing: True | |
| only_pretrain_model: False | |
| cpu_ram_mb: 32088 | |
| use_gpu: True | |
| num_gpus: 1 | |
| gpu: TITAN RTX | |
| gpu_ram_mb: 24217 | |
| gpu_power_watts: 280.0 | |
| gpu_performance_state: 2 | |
| use_tpu: False | |
| By default, the time and the required memory for inference are benchmarked. In the example output above the first | |
| two sections show the result corresponding to inference time and inference memory. In addition, all relevant | |
| information about the computing environment, e.g. the GPU type, the system, the library versions, etc are printed | |
| out in the third section under ENVIRONMENT INFORMATION. This information can optionally be saved in a .csv file | |
| when adding the argument save_to_csv=True to [PyTorchBenchmarkArguments] and | |
| [TensorFlowBenchmarkArguments] respectively. In this case, every section is saved in a separate | |
| .csv file. The path to each .csv file can optionally be defined via the argument data classes. | |
| Instead of benchmarking pre-trained models via their model identifier, e.g. google-bert/bert-base-uncased, the user can | |
| alternatively benchmark an arbitrary configuration of any available model class. In this case, a list of | |
| configurations must be inserted with the benchmark args as follows. | |
| from transformers import PyTorchBenchmark, PyTorchBenchmarkArguments, BertConfig | |
| args = PyTorchBenchmarkArguments( | |
| models=["bert-base", "bert-384-hid", "bert-6-lay"], batch_sizes=[8], sequence_lengths=[8, 32, 128, 512] | |
| ) | |
| config_base = BertConfig() | |
| config_384_hid = BertConfig(hidden_size=384) | |
| config_6_lay = BertConfig(num_hidden_layers=6) | |
| benchmark = PyTorchBenchmark(args, configs=[config_base, config_384_hid, config_6_lay]) | |
| benchmark.run() | |
| ==================== INFERENCE - SPEED - RESULT ==================== |