Victarry commited on
Commit
4f8da72
·
1 Parent(s): c048b97

Update readme.

Browse files
Files changed (1) hide show
  1. README.md +39 -2
README.md CHANGED
@@ -1,9 +1,46 @@
1
- # Pipeline Parallelism Emulation and Visualization
 
 
 
 
 
 
 
 
 
2
 
3
- This project provides tools for emulating and visualizing pipeline parallelism strategies used in large language model training.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
 
5
  ## Overview
6
 
 
 
7
  Pipeline parallelism is a technique used to train large models by partitioning the model across multiple devices and processing data in a pipelined fashion. This project allows you to:
8
 
9
  - Simulate different pipeline parallelism strategies (1F1B, Interleaved, Zero-Bubble, etc.)
 
1
+ ---
2
+ title: Pipeline Parallelism Schedule Visualizer
3
+ emoji: 📊
4
+ colorFrom: indigo
5
+ colorTo: blue
6
+ sdk: docker
7
+ sdk_version: latest
8
+ app_file: app.py
9
+ pinned: false
10
+ ---
11
 
12
+ # Pipeline Parallelism Schedule Visualizer
13
+
14
+ An interactive visualization tool for exploring different pipeline parallelism scheduling strategies in large language models.
15
+
16
+ ## Features
17
+
18
+ - Visualize multiple scheduling strategies for pipeline parallelism
19
+ - Adjust parameters like number of devices, stages, and batches
20
+ - Compare execution timelines between different strategies
21
+ - Explore operation timings and their effects on performance
22
+
23
+ ## Supported Strategies
24
+
25
+ - 1F1B (One-Forward-One-Backward)
26
+ - 1F1B with Interleaved Placement
27
+ - 1F1B with Overlapped Operations
28
+ - 1F1B with Interleaved Placement and Overlapped Operations
29
+ - Zero-Bubble 1 Pipeline (ZB1P)
30
+ - Dual Pipeline (DualPipe)
31
+
32
+ ## Usage
33
+
34
+ Simply adjust the parameters and select the strategies you want to compare, then click "Generate Schedule" to visualize the results.
35
+
36
+ ## Deployment
37
+
38
+ This app is deployed on Hugging Face Spaces using Dash.
39
 
40
  ## Overview
41
 
42
+ This project provides tools for emulating and visualizing pipeline parallelism strategies used in large language model training.
43
+
44
  Pipeline parallelism is a technique used to train large models by partitioning the model across multiple devices and processing data in a pipelined fashion. This project allows you to:
45
 
46
  - Simulate different pipeline parallelism strategies (1F1B, Interleaved, Zero-Bubble, etc.)