Spaces:
Running
on
Zero
Apply for community grant: Personal project (gpu)
I'm applying for a GPU grant in my project named DepthFlow (704 ⭐️) - it converts "Images to 3D Parallax Video" using their estimated depth maps, often via state of the art monocular depth estimation models like DepthAnything2. Gradio itself has shared it in X/Twitter and Linkedin in the past! It is my current oldest unresolved issue in GitHub having it hosted on spaces, and a recurrent user request for a website.
Speeds: The processing is done in an optimized GLSL Shader, achieving 170fps at 4k resolution on an RTX 3060 in stress tests. The main bottleneck in production usage (exporting videos) is the encoding itself, which I can swap for NVENC to do the job instead of the CPU if available.
Suggested GPU: I believe the cheapest HuggingFace's option for GPU spaces, the t4-small, should be plenty for me - as NVENC doesn't doubles throughput in higher tier cards, plus CUDA is only briefly used for MDE. Albeit t4-medium would be preferable if there's multiple simultaneous jobs, as a single render uses roughly 3x to 4x maxed out theads offloaded to NVENC:
$ /usr/bin/time -v depthflow h264-nvenc main -o /tmp/video.mp4 --time 60
(...)
User time (seconds): 34.51
System time (seconds): 8.51
Percent of CPU this job got: 338%
ZeroGPU also cuts it, though sub-utilizing the A100's resources for simple encoding and lightweight CUDA/VRAM tasks.
Technical concerns: I will need graphics capabilities enabled in NVIDIA Container Toolkit on the host for EGL to find the GPU as a OpenGL capable device in headless rendering, if not already enabled (and some packages, configuration on my side). Some cloud providers seem to disable NVENC due security concerns, not sure if it's the case here, better safe than sorry!
Sidenote: This configuration for EGL could be baked-in your builds, it's non-trivial getting it working :)
A clarification on "Personal project" vs hosting on a organization: I am the current sole maintainer with no legal company behind it, it's mostly for the separation of personal (lower-tier) and professional projects, making it easier to manage potential contributors. I may be a company in the future, if time comes I'll be sure to notify, but for the medium term, it's personal.
Forgot to showcase how slow it is on pure CPU with software rendering (llvmpipe device) 😅
Hi @Tremeschin , we've assigned ZeroGPU to this Space. Please check the compatibility and usage sections of this page so your Space can run on ZeroGPU.
Thanks! I'm working on the changes required for it to run in the environment, most notably facing some pickling errors but I can fix them.
Hi again
@hysts
, after fixing minor Python 3.10 compatibilities, ensuring my .upscale, .estimate and .render methods are wrapped by spaces.GPU and have pickable arguments, creating the required Installable Client Drivers (ICD) files for EGL and Vulkan, the worst possible outcome happened as in the image below 😓
Issues: There are some configurations not reachable by me:
- The
graphicscapability doesn't seem to be enabled on the runner's NVIDIA Container Toolkit, so it fallbacks to CPU rendering withllvmpipeand is very slow. - The NVIDIA driver is unsupported by FFmpeg's NVENC implementation.
Help needed: Could you guys from HuggingFace's side help me further here?
- Enabling
graphicscapabilities on the runner for OpenGL / Vulkan applications - Having at least drivers
470.57.02as FFmpeg says so, ideally the most up to date one
Thanks!
(Edit: Comment below was duped, clicked post comment twice while my internet was out 🤦♂️)
Let me cc @cbensimon
Hi, sorry for the re-ping @hysts @cbensimon , it's been a month and both EGL and NVENC aren't available on the Space, raising errors for any kind of usage. Got 386 all time visits and 220 ZeroGPU runs since then and a few people are complaining in my Issue/DMs, could you revisit my case? Thanks! 🙂
Hi
@Tremeschin
.
Indeed, ZeroGPU only enables compute,utility capabilities.
Do you know which hardware unit(s) (among NVDEC, NVJPG and OFA) is needed for your use case?
Hello @cbensimon
Do you know which hardware unit(s) (among NVDEC, NVJPG and OFA) is needed for your use case?
I would only need NVENC, as the project does: OpenGL Shader => Raw data => FFmpeg => Video
Just to better utilize the A100 on ZeroGPU, as CPU encoding falls miles behind what the GPU can render.
Indeed, ZeroGPU only enables compute,utility capabilities.
Ah, I'd really need graphics and maybe video (nvenc?), else I may close the space for being infeasible :(
The GPU isn't being used for the shader, and CPU rendering is impractically slow. I have this GitHub Comment talking about what needs to happen for EGL to find the GPU with a minimal reproducible example, if you want to take a deeper look or test it.
I know there could be security implications with such, but could it be enabled in my special case?
Thanks for your time!