Spaces:
Running
Running
Update readme.
Browse files
README.md
CHANGED
@@ -1,9 +1,46 @@
|
|
1 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2 |
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
4 |
|
5 |
## Overview
|
6 |
|
|
|
|
|
7 |
Pipeline parallelism is a technique used to train large models by partitioning the model across multiple devices and processing data in a pipelined fashion. This project allows you to:
|
8 |
|
9 |
- Simulate different pipeline parallelism strategies (1F1B, Interleaved, Zero-Bubble, etc.)
|
|
|
1 |
+
---
|
2 |
+
title: Pipeline Parallelism Schedule Visualizer
|
3 |
+
emoji: 📊
|
4 |
+
colorFrom: indigo
|
5 |
+
colorTo: blue
|
6 |
+
sdk: docker
|
7 |
+
sdk_version: latest
|
8 |
+
app_file: app.py
|
9 |
+
pinned: false
|
10 |
+
---
|
11 |
|
12 |
+
# Pipeline Parallelism Schedule Visualizer
|
13 |
+
|
14 |
+
An interactive visualization tool for exploring different pipeline parallelism scheduling strategies in large language models.
|
15 |
+
|
16 |
+
## Features
|
17 |
+
|
18 |
+
- Visualize multiple scheduling strategies for pipeline parallelism
|
19 |
+
- Adjust parameters like number of devices, stages, and batches
|
20 |
+
- Compare execution timelines between different strategies
|
21 |
+
- Explore operation timings and their effects on performance
|
22 |
+
|
23 |
+
## Supported Strategies
|
24 |
+
|
25 |
+
- 1F1B (One-Forward-One-Backward)
|
26 |
+
- 1F1B with Interleaved Placement
|
27 |
+
- 1F1B with Overlapped Operations
|
28 |
+
- 1F1B with Interleaved Placement and Overlapped Operations
|
29 |
+
- Zero-Bubble 1 Pipeline (ZB1P)
|
30 |
+
- Dual Pipeline (DualPipe)
|
31 |
+
|
32 |
+
## Usage
|
33 |
+
|
34 |
+
Simply adjust the parameters and select the strategies you want to compare, then click "Generate Schedule" to visualize the results.
|
35 |
+
|
36 |
+
## Deployment
|
37 |
+
|
38 |
+
This app is deployed on Hugging Face Spaces using Dash.
|
39 |
|
40 |
## Overview
|
41 |
|
42 |
+
This project provides tools for emulating and visualizing pipeline parallelism strategies used in large language model training.
|
43 |
+
|
44 |
Pipeline parallelism is a technique used to train large models by partitioning the model across multiple devices and processing data in a pipelined fashion. This project allows you to:
|
45 |
|
46 |
- Simulate different pipeline parallelism strategies (1F1B, Interleaved, Zero-Bubble, etc.)
|