GPU on Producthunt daily

Sunshine

Tue, 23 Sep 2025 15:28:47 +0800

LizardByte/Sunshine

Sunshine

Self-hosted game stream host for Moonlight.

ℹ️ About

Sunshine is a self-hosted game stream host for Moonlight. Offering low latency, cloud gaming server capabilities with support for AMD, Intel, and Nvidia GPUs for hardware encoding. Software encoding is also available. You can connect to Sunshine from any Moonlight client on a variety of devices. A web UI is provided to allow configuration, and client pairing, from your favorite web browser. Pair from the local server or any mobile device.

LizardByte has the full documentation hosted on Read the Docs

Stable
Beta

🖥️ System Requirements

[!WARNING] These tables are a work in progress. Do not purchase hardware based on this information.

Minimum Requirements
Component	Requirement
GPU	AMD: VCE 1.0 or higher, see: obs-amd hardware support
Intel: Linux: VAAPI-compatible, see: VAAPI hardware support Windows: Skylake or newer with QuickSync encoding support
Nvidia: NVENC enabled cards, see: nvenc support matrix
CPU	AMD: Ryzen 3 or higher
Intel: Core i3 or higher
RAM	4GB or more
OS	Windows: 10+ (Windows Server does not support virtual gamepads)
macOS: 14+
Linux/Debian: 13+ (trixie)
Linux/Fedora: 41+
Linux/Ubuntu: 22.04+ (jammy)
Network	Host: 5GHz, 802.11ac
Client: 5GHz, 802.11ac

4k Suggestions
Component	Requirement
GPU	AMD: Video Coding Engine 3.1 or higher
Intel: Linux: HD Graphics 510 or higher Windows: Skylake or newer with QuickSync encoding support
Nvidia: Linux: GeForce RTX 2000 series or higher Windows: Geforce GTX 1080 or higher
CPU	AMD: Ryzen 5 or higher
Intel: Core i5 or higher
Network	Host: CAT5e ethernet or better
Client: CAT5e ethernet or better

HDR Suggestions
Component	Requirement
GPU	AMD: Video Coding Engine 3.4 or higher
Intel: HD Graphics 730 or higher
Nvidia: Pascal-based GPU (GTX 10-series) or higher
CPU	AMD: Ryzen 5 or higher
Intel: Core i5 or higher
Network	Host: CAT5e ethernet or better
Client: CAT5e ethernet or better

❓ Support

Our support methods are listed in our LizardByte Docs.

💲 Sponsors and Supporters

👥 Contributors

Thank you to all the contributors who have helped make Sunshine better!

GitHub

CrowdIn

Previous	Next
	Getting Started

[TOC]

LMCache

Wed, 20 Aug 2025 15:28:48 +0800

LMCache/LMCache

🔥 NEW: For enterprise-scale deployment of LMCache and vLLM, please check out vLLM Production Stack. LMCache is also officially supported in llm-d and KServe!

Summary

LMCache is an LLM serving engine extension to reduce TTFT and increase throughput, especially under long-context scenarios. By storing the KV caches of reusable texts across various locations, including (GPU, CPU DRAM, Local Disk), LMCache reuses the KV caches of any reused text (not necessarily prefix) in any serving engine instance. Thus, LMCache saves precious GPU cycles and reduces user response delay.

By combining LMCache with vLLM, developers achieve 3-10x delay savings and GPU cycle reduction in many LLM use cases, including multi-round QA and RAG.

Features

🔥 Integration with vLLM v1 with the following features:
- High performance CPU KVCache offloading
- Disaggregated prefill
- P2P KVCache sharing
LMCache is supported in the vLLM production stack, llm-d, and KServe
Stable support for non-prefix KV caches
Storage support as follows:
- CPU
- Disk
- NIXL
Installation support through pip and latest vLLM

Installation

To use LMCache, simply install lmcache from your package manager, e.g. pip:

`1`	`pip install lmcache`

Works on Linux NVIDIA GPU platform.

More detailed installation instructions are available in the docs, particularly if you are not using the latest stable version of vllm or using another serving engine with different dependencies. Any “undefined symbol” or torch mismatch versions can be resolved in the documentation.

Getting started

The best way to get started is to checkout the Quickstart Examples in the docs.

Documentation

Check out the LMCache documentation which is available online.

We also post regularly in LMCache blogs.

Examples

Go hands-on with our examples, demonstrating how to address different use cases with LMCache.

Interested in Connecting?

Fill out the interest form, sign up for our newsletter, join LMCache slack, check out LMCache website, or drop an email, and our team will reach out to you!

Community meeting

The community meeting for LMCache is hosted bi-weekly. All are welcome to join!

Meetings are held bi-weekly on: Tuesdays at 9:00 AM PT – Add to Calendar

We keep notes from each meeting on this document for summaries of standups, discussion, and action items.

Recordings of meetings are available on the YouTube LMCache channel.

Contributing

We welcome and value all contributions and collaborations. Please check out Contributing Guide on how to contribute.

We continually update [Onboarding] Welcoming contributors with good first issues!

Citation

If you use LMCache for your research, please cite our papers:

@inproceedings{liu2024cachegen,
  title={Cachegen: Kv cache compression and streaming for fast large language model serving},
  author={Liu, Yuhan and Li, Hanchen and Cheng, Yihua and Ray, Siddhant and Huang, Yuyang and Zhang, Qizheng and Du, Kuntai and Yao, Jiayi and Lu, Shan and Ananthanarayanan, Ganesh and others},
  booktitle={Proceedings of the ACM SIGCOMM 2024 Conference},
  pages={38--56},
  year={2024}
}

@article{cheng2024large,
  title={Do Large Language Models Need a Content Delivery Network?},
  author={Cheng, Yihua and Du, Kuntai and Yao, Jiayi and Jiang, Junchen},
  journal={arXiv preprint arXiv:2409.13761},
  year={2024}
}

@inproceedings{10.1145/3689031.3696098,
  author = {Yao, Jiayi and Li, Hanchen and Liu, Yuhan and Ray, Siddhant and Cheng, Yihua and Zhang, Qizheng and Du, Kuntai and Lu, Shan and Jiang, Junchen},
  title = {CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion},
  year = {2025},
  url = {https://doi.org/10.1145/3689031.3696098},
  doi = {10.1145/3689031.3696098},
  booktitle = {Proceedings of the Twentieth European Conference on Computer Systems},
  pages = {94–109},
}

Socials

Linkedin | Twitter | Youtube

License

The LMCache codebase is licensed under Apache License 2.0. See the LICENSE file for details.

hashcat

Wed, 06 Aug 2025 15:37:25 +0800

hashcat/hashcat

hashcat

hashcat is the world’s fastest and most advanced password recovery utility, supporting five unique modes of attack for over 300 highly-optimized hashing algorithms. hashcat currently supports CPUs, GPUs, and other hardware accelerators on Linux, Windows, and macOS, and has facilities to help enable distributed password cracking.

License

hashcat is licensed under the MIT license. Refer to docs/license.txt for more information.

Installation

Download the latest release and unpack it in the desired location. Please remember to use 7z x when unpacking the archive from the command line to ensure full file paths remain intact.

Usage/Help

Please refer to the Hashcat Wiki and the output of --help for usage information and general help. A list of frequently asked questions may also be found here. The Hashcat Forum also contains a plethora of information. If you still think you need help by a real human come to Discord.

Building

Refer to BUILD.md for instructions on how to build hashcat from source.

Tests:

Travis	Coverity	GitHub Actions

Contributing

Contributions are welcome and encouraged, provided your code is of sufficient quality. Before submitting a pull request, please ensure your code adheres to the following requirements:

Licensed under MIT license, or dedicated to the public domain (BSD, GPL, etc. code is incompatible)
Adheres to gnu99 standard
Compiles cleanly with no warnings when compiled with -W -Wall -std=gnu99
Uses Allman-style code blocks & indentation
Uses 2-spaces as the indentation or a tab if it’s required (for example: Makefiles)
Uses lower-case function and variable names
Avoids the use of ! and uses positive conditionals wherever possible (e.g., if (foo == 0) instead of if (!foo), and if (foo) instead of if (foo != 0))
Use code like array[index + 0] if you also need to do array[index + 1], to keep it aligned

You can use GNU Indent to help assist you with the style requirements:

`1`	`indent -st -bad -bap -sc -bl -bli0 -ncdw -nce -cli0 -cbi0 -pcs -cs -npsl -bs -nbc -bls -blf -lp -i2 -ts2 -nut -l1024 -nbbo -fca -lc1024 -fc1`

Your pull request should fully describe the functionality you are adding/removing or the problem you are solving. Regardless of whether your patch modifies one line or one thousand lines, you must describe what has prompted and/or motivated the change.

Solve only one problem in each pull request. If you’re fixing a bug and adding a new feature, you need to make two separate pull requests. If you’re fixing three bugs, you need to make three separate pull requests. If you’re adding four new features, you need to make four separate pull requests. So on, and so forth.

If your patch fixes a bug, please be sure there is an issue open for the bug before submitting a pull request. If your patch aims to improve performance or optimize an algorithm, be sure to quantify your optimizations and document the trade-offs, and back up your claims with benchmarks and metrics.

In order to maintain the quality and integrity of the hashcat source tree, all pull requests must be reviewed and signed off by at least two board members before being merged. The project lead has the ultimate authority in deciding whether to accept or reject a pull request. Do not be discouraged if your pull request is rejected!

Happy Cracking!

ZLUDA

Mon, 07 Jul 2025 15:31:03 +0800

vosen/ZLUDA

ZLUDA

ZLUDA is a drop-in replacement for CUDA on non-NVIDIA GPU. ZLUDA allows to run unmodified CUDA applications using non-NVIDIA GPUs with near-native performance.

ZLUDA supports AMD Radeon RX 5000 series and newer GPUs (both desktop and integrated).

ZLUDA is work in progress. Follow development here and say hi on Discord. For more details see the announcement: https://vosen.github.io/ZLUDA/blog/zludas-third-life/

Usage

Warning: This version ZLUDA is under heavy development (more here) and right now only supports Geekbench. ZLUDA probably will not work with your application just yet.

Windows

You should have recent AMD GPU driver (“AMD Software: Adrenalin Edition”) installed.
To run your application you should etiher:

(Recommended approach) Copy ZLUDA-provided nvcuda.dll and nvml.dll from target\release (if built from sources) or zluda (if downloaded a zip package) into a path which your application uses to load CUDA. Paths vary application to application, but usually it’s the directory where the .exe file is located
Use ZLUDA launcher like below. ZLUDA launcher is known to be buggy and incomplete:
1

<ZLUDA_DIRECTORY>\zluda_with.exe -- <APPLICATION> <APPLICATIONS_ARGUMENTS>

Linux

Run your application like this:

`1`	`LD_LIBRARY_PATH=<ZLUDA_DIRECTORY> <APPLICATION> <APPLICATIONS_ARGUMENTS>`

where <ZLUDA_DIRECTORY> is the directory which contains ZLUDA-provided libcuda.so: target/release if you built from sources or zluda if you downloaded prebuilt package.

MacOS

Not supported

Building

Dependencies

Git
CMake
Python 3
Rust compiler (recent version)
C++ compiler
(Optional, but recommended) Ninja build system

Build steps

Git clone the repo (make sure to use --recursive option to fetch submodules):
git clone --recursive https://github.com/vosen/ZLUDA.git
Enter freshly cloned ZLUDA directory and build with cargo (this takes a while):
cargo xtask --release

Contributing

ZLUDA project has a commercial backing and does not accept donations. ZLUDA project accepts pull requests and other non-monetary contributions.

If you want to contribute a code fix or documentation update feel free to open a Pull Request.

Getting started

There’s no architecture document (yet). Two most important crates in ZLUDA are ptx (PTX compiler) and zluda (AMD GPU runtime). A good starting point to tinkering the project is to run one of the ptx unit tests under a debugger and understand what it is doing. cargo test -p ptx -- ::add_hip is a simple test that adds two numbers.

Github issues tagged with “help wanted” are tasks that are self-containted. Their level of difficulty varies, they are not always good beginner tasks, but they defined unambiguously.

If you have questions feel free to ask on #devtalk channel on Discord.

License

This software is dual-licensed under either the Apache 2.0 license or the MIT license. See LICENSE-APACHE or LICENSE-MIT for details

rl-swarm

Sat, 28 Jun 2025 15:28:05 +0800

gensyn-ai/rl-swarm

RL Swarm

RL Swarm is a peer-to-peer system for reinforcement learning. It allows you to train models collaboratively with others in the swarm, leveraging their collective intelligence. It is open source and permissionless, meaning you can run it on a consumer laptop at home or on a powerful GPU in the cloud. You can also connect your model to the Gensyn Testnet to receive an on-chain identity that tracks your progress over time.

Currently, we are running the reasoning-gym swarm on the Testnet. This swarm is designed to train models to solve a diverse set of reasoning tasks using the reasoning-gym dataset. The current list of default models includes:

Models:

Gensyn/Qwen2.5-0.5B-Instruct
Qwen/Qwen3-0.6B
nvidia/AceInstruct-1.5B
dnotitia/Smoothie-Qwen3-1.7B
Gensyn/Qwen2.5-1.5B-Instruct

This iteration of rl-swarm is powered by the GenRL-Swarm library. It is a fully composable framework for decentralized reinforcement learning which enables users to create and customize their own swarms for reinforcement learning with multi-agent multi-stage environments.

Requirements

Your hardware requirements will vary depending on a number of factors including model size and the accelerator platform you use. Users running large NVIDIA GPU will be assigned a model from the large model pool, while users running less powerful hardware will be assigned a model from the small model pool. This design decision is intended to allow users to advance at a similar rate regardless of the hardware they use, maximizing their utility to the swarm.

Supported Hardware

arm64 or x86 CPU with minimum 32gb ram (note that if you run other applications during training it might crash training).

CUDA devices (officially supported):
- RTX 3090
- RTX 4090
- RTX 5090
- A100
- H100

With either configuration, you will need Python >=3.10 (for Mac, you will likely need to upgrade).

⚠️ Please read before continuing ⚠️

This software is experimental and provided as-is for users who are interested in using (or helping to develop) an early version of the Gensyn Protocol for training models.

If you care about on-chain participation, you must read the Identity Management section below.

If you encounter issues, please first check Troubleshooting. If you cannot find a solution there, please check if there is an open (or closed) Issue. If there is no relevant issue, please file one and include 1) all relevant logs, 2) information about your device (e.g. which GPU, if relevant), and 3) your operating system information.

Instructions

Run the Swarm

The easiest way to run RL Swarm is using Docker. This ensures a consistent setup across all operating systems with minimal dependencies.

1. Clone this repo

`1`	`git clone https://github.com/gensyn-ai/rl-swarm`

2. Install Docker

Make sure you have Docker installed and the Docker daemon is running on your machine. To do that, follow these instructions according to your OS. Ensure you allot sufficient memory to the Docker containers. For example if using Docker Desktop, this can be done by going to Docker Desktop Settings > Resources > Advanced > Memory Limit, and increasing it to the maximum possible value.

3. Start the Swarm

Run the following commands from the root of the repository.

CPU support

If you’re using a Mac or if your machine has CPU-only support:

`1`	`docker-compose run --rm --build -Pit swarm-cpu`

GPU support

If you’re using a machine with an officially supported GPU:

`1`	`docker-compose run --rm --build -Pit swarm-gpu`

Docker compose issue

If docker-compose does not work when running the above commands, please try docker compose (no hyphen) instead. I.e. docker compose run --rm --build -Pit swarm-gpu. This issue sometimes occurs on users running Ubuntu.

Experimental (advanced) mode

If you want to experiment with the GenRL-Swarm library and its configurable parameters, we recommend you run RL Swarm via shell script:

1
2
3

python3 -m venv .venv
source .venv/bin/activate
./run_rl_swarm.sh

To learn more about experimental mode, check out our getting started guide.

A browser window will pop open (you’ll need to manually navigate to http://localhost:3000/ if you’re on a VM).
Click ’login'.
Login with your preferred method.

Huggingface

If you would like to upload your model to Hugging Face, enter your Hugging Face access token when prompted. You can generate one from your Hugging Face account, under Access Tokens.

Initial peering and training

From this stage onward your device will begin training. You should see your peer register and vote on-chain here.

You can also track your training progress in real time:

On The RL-Swarm Dashboard: dashboard.gensyn.ai

Identity management

Introduction

On-chain identity is managed via an Alchemy modal sign-in screen. You need to supply an email address or login via a supported method (e.g. Google). This creates an EOA public/private key (which are stored by Alchemy). You will also receive local session keys in the userApiKey. Note that these aren’t your EOA public/private keys.

During the initial set-up process, you will also create a swarm.pem file which maintains the identity of your peer. This is then registered on chain using the EOA wallet hosted in Alchemy, triggered using your local api keys. This links the swarm.pem to the email address (and corresponding EOA in Alchemy).

If you want to link multiple nodes to a single EOA, simply sign up each node using the same email address. You will get a new peer ID for each node, however they will all be linked to the same EOA that your email is linked to.

Please note: if you are using a fork of this repo, or a service organised by someone else (e.g. a ‘one click deployment’ provider) the identity management flow below is not guaranteed.

What this means

In the following two scenarios, everything will work (i.e. you will have an on-chain identity linked with your RL Swarm peer training):

The very first time you run the node from scratch with a new email address. The smart account will be created fresh and linked with the swarm.pem that is also fresh.
If you run it again with a swarm.pem AND login the original email address used with that swarm.pem. Note: this will throw an error into the log on registration but will still be able to sign transactions.

In the following two scenarios, it will not work (i.e. you won’t have an on-chain identity linked with your RL Swarm peer training):

If you keep your swarm.pem and try to link it to an email address distinct from the one with which it was first registered.

Therefore, you should do these actions in the following scenarios

Signed up with email address, generated swarm.pem, BUT lost swarm.pem OR You want to run multiple nodes at once: run from scratch with the same email address and generate a new swarm.pem.
Signed up with email address, generated swarm.pem, kept swarm.pem -> you can re-run a single node using this pair if you’ve still got them both.

Troubleshooting

How do I find my logs? You can find them inside the /logs directory:
- yarn.log: This file contains logs for the modal login server.
- swarm.log: This is the main log file for the RL Swarm application.
- wandb/: This directory contains various logs related to your training runs, including a debug.log file. These can be updated to Weights & Biases (only available if you log_with wandb).
My peer ‘skipped a round’: this occurs when your device isn’t fast enough to keep up with the pace of the swarm. For example, if you start training at round 100 and by the time you finish training the rest of the swarm reaches round 102, you will skip round 101 and go straight to 102. This is because your peer is more valuable if it is participating in the active round.
My model doesn’t seem to be training?
- If you’re using a consumer device (e.g. a MacBook), it is likely just running slowly - check back in 20 minutes.
Logging in with a new account after previous login?
- Make sure you click ‘Logout’ on the login screen before you leave your previous session
- Make sure you delete swarm.pem from the root directory (try sudo rm swarm.pem). If you don’t do this, and you previously registered with the peer-id stored in this file, it will disrupt the training process.
Issues with the Login screen
- Upgrade viem: some users report issues with the viem package. There are two fixes:
  - in the modal-login/package.json update: "viem": "2.25.0"
  - in the terminal cd /root/rl-swarm/modal-login/ && yarn upgrade && yarn add next@latest && yarn add viem@latest
I’m getting lots of warnings
- This is expected behaviour and usually the output of the package managers or other dependencies. The most common is the below Protobuf warning - which can be ignored
  1
  
  WARNING: The candidate selected for download or install is a yanked version: 'protobuf' candidate...
Issues on VMs/VPSs?
- How do I access the login screen if I’m running in a VM?: port forwarding. Add this SSH flag: -L 3000:localhost:3000 when connecting to your VM. E.g. gcloud compute ssh --zone "us-central1-a" [your-vm] --project [your-project] -- -L 3000:localhost:3000. Note, some VPSs may not work with rl-swarm. Check the Gensyn discord for up-to-date information on this.
- Disconnection/general issues: If you are tunneling to a VM and suffer a broken pipe, you will likely encounter OOM or unexpected behaviour the first time you relaunch the script. If you control + c and kill the script it should spin down all background processes. Restart the script and everything should work normally.
Issues with npm/general installation?
- Try npm install -g node@latest
OOM errors on MacBook?
- Try this (experimental) fix to increase memory:
  1
  
  export PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0
I have a Windows machine, can I still train a model on the swarm?: Yes - but this is not very well tested and may require you to do some debugging to get it set up properly. Install WSL and Linux on your Windows machine using the following instructions: https://learn.microsoft.com/en-us/windows/wsl/install
I want to move my to a different machine and/or restart with a fresh build of the repo, but I want my animal name/peer id to persist.: To achieve this simply backup the swarm.pem file on your current machine and then put it in the corresponding location on your new machine/build of the repo.
I have multiple GPUs on one machine, can I run multiple peers?: Yes - but you’ll need to manually change things. You’ll need to isolate each GPU, install this repo for each GPU, and expose each peer under a different port to pass the modal onboard.
My round/stage is behind the smart contract/other peers?: This is expected behaviour given the different speeds of machines in the network. Once your machine completes it’s current round, it will move to the the current round.
I want to use a bigger and/or different model in the RL swarm, can I do that?: Yes - but we only recommend doing so if you are comfortable understanding what size model can reasonably run on your hardware. If you elect to bring a custom model, just paste the repo/model name into the command line when prompted.
I am running a model in the swarm on my CPU, have received a python RuntimeError, and my training progress seems to have stopped.: There are several possible causes for this, but before trying anything please wait long enough to be sure your training actually is frozen and not just slow (e.g., wait longer than a single training iteration has previously taken on your machine). If you’re sure training is actually frozen, then some things to try are:
- Set this (experimental) fix: export PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 && ./run_rl_swarm.sh

tinygrad

Wed, 21 May 2025 15:30:12 +0800

tinygrad/tinygrad

tinygrad: For something between PyTorch and karpathy/micrograd. Maintained by tiny corp.

Homepage | Documentation | Discord

This may not be the best deep learning framework, but it is a deep learning framework.

Due to its extreme simplicity, it aims to be the easiest framework to add new accelerators to, with support for both inference and training. If XLA is CISC, tinygrad is RISC.

tinygrad is still alpha software, but we raised some money to make it good. Someday, we will tape out chips.

Features

LLaMA and Stable Diffusion

tinygrad can run LLaMA and Stable Diffusion!

Laziness

Try a matmul. See how, despite the style, it is fused into one kernel with the power of laziness.

DEBUG=3 python3 -c "from tinygrad import Tensor;
N = 1024; a, b = Tensor.rand(N, N), Tensor.rand(N, N);
c = (a.reshape(N, 1, N) * b.T.reshape(1, N, N)).sum(axis=2);
print((c.numpy() - (a.numpy() @ b.numpy())).mean())"

And we can change DEBUG to 4 to see the generated code.

Neural networks

As it turns out, 90% of what you need for neural networks are a decent autograd/tensor library. Throw in an optimizer, a data loader, and some compute, and you have all you need.

from tinygrad import Tensor, nn

class LinearNet:
  def __init__(self):
    self.l1 = Tensor.kaiming_uniform(784, 128)
    self.l2 = Tensor.kaiming_uniform(128, 10)
  def __call__(self, x:Tensor) -> Tensor:
    return x.flatten(1).dot(self.l1).relu().dot(self.l2)

model = LinearNet()
optim = nn.optim.Adam([model.l1, model.l2], lr=0.001)

x, y = Tensor.rand(4, 1, 28, 28), Tensor([2,4,3,7])  # replace with real mnist dataloader

with Tensor.train():
  for i in range(10):
    optim.zero_grad()
    loss = model(x).sparse_categorical_crossentropy(y).backward()
    optim.step()
    print(i, loss.item())

See examples/beautiful_mnist.py for the full version that gets 98% in ~5 seconds

Accelerators

tinygrad already supports numerous accelerators, including:

And it is easy to add more! Your accelerator of choice only needs to support a total of ~25 low level ops.

To check default accelerator run: python3 -c "from tinygrad import Device; print(Device.DEFAULT)"

Installation

The current recommended way to install tinygrad is from source.

From source

1
2
3

git clone https://github.com/tinygrad/tinygrad.git
cd tinygrad
python3 -m pip install -e .

Direct (master)

`1`	`python3 -m pip install git+https://github.com/tinygrad/tinygrad.git`

Documentation

Documentation along with a quick start guide can be found on the docs website built from the docs/ directory.

Quick example comparing to PyTorch

from tinygrad import Tensor

x = Tensor.eye(3, requires_grad=True)
y = Tensor([[2.0,0,-2.0]], requires_grad=True)
z = y.matmul(x).sum()
z.backward()

print(x.grad.tolist())  # dz/dx
print(y.grad.tolist())  # dz/dy

The same thing but in PyTorch:

import torch

x = torch.eye(3, requires_grad=True)
y = torch.tensor([[2.0,0,-2.0]], requires_grad=True)
z = y.matmul(x).sum()
z.backward()

print(x.grad.tolist())  # dz/dx
print(y.grad.tolist())  # dz/dy

Contributing

There has been a lot of interest in tinygrad lately. Following these guidelines will help your PR get accepted.

We’ll start with what will get your PR closed with a pointer to this section:

No code golf! While low line count is a guiding light of this project, anything that remotely looks like code golf will be closed. The true goal is reducing complexity and increasing readability, and deleting \ns does nothing to help with that.
All docs and whitespace changes will be closed unless you are a well-known contributor. The people writing the docs should be those who know the codebase the absolute best. People who have not demonstrated that shouldn’t be messing with docs. Whitespace changes are both useless and carry a risk of introducing bugs.
Anything you claim is a “speedup” must be benchmarked. In general, the goal is simplicity, so even if your PR makes things marginally faster, you have to consider the tradeoff with maintainability and readability.
In general, the code outside the core tinygrad/ folder is not well tested, so unless the current code there is broken, you shouldn’t be changing it.
If your PR looks “complex”, is a big diff, or adds lots of lines, it won’t be reviewed or merged. Consider breaking it up into smaller PRs that are individually clear wins. A common pattern I see is prerequisite refactors before adding new functionality. If you can (cleanly) refactor to the point that the feature is a 3 line change, this is great, and something easy for us to review.

Now, what we want:

Bug fixes (with a regression test) are great! This library isn’t 1.0 yet, so if you stumble upon a bug, fix it, write a test, and submit a PR, this is valuable work.
Solving bounties! tinygrad offers cash bounties for certain improvements to the library. All new code should be high quality and well tested.
Features. However, if you are adding a feature, consider the line tradeoff. If it’s 3 lines, there’s less of a bar of usefulness it has to meet over something that’s 30 or 300 lines. All features must have regression tests. In general with no other constraints, your feature’s API should match torch or numpy.
Refactors that are clear wins. In general, if your refactor isn’t a clear win it will be closed. But some refactors are amazing! Think about readability in a deep core sense. A whitespace change or moving a few functions around is useless, but if you realize that two 100 line functions can actually use the same 110 line function with arguments while also improving readability, this is a big win. Refactors should pass process replay.
Tests/fuzzers. If you can add tests that are non brittle, they are welcome. We have some fuzzers in here too, and there’s a plethora of bugs that can be found with them and by improving them. Finding bugs, even writing broken tests (that should pass) with @unittest.expectedFailure is great. This is how we make progress.
Dead code removal from core tinygrad/ folder. We don’t care about the code in extra, but removing dead code from the core library is great. Less for new people to read and be confused by.

Running tests

You should install the pre-commit hooks with pre-commit install. This will run the linter, mypy, and a subset of the tests on every commit.

For more examples on how to run the full test suite please refer to the CI workflow.

Some examples of running tests locally:

1
2
3

python3 -m pip install -e '.[testing]'  # install extra deps for testing
python3 test/test_ops.py                # just the ops tests
python3 -m pytest test/                 # whole test suite

Process replay tests

Process replay compares your PR’s generated kernels against master. If your PR is a refactor or speedup without any expected behavior change, It should include [pr] in the pull request title.

skypilot

Sun, 27 Apr 2025 15:27:09 +0800

skypilot-org/skypilot

Run AI on Any Infra — Unified, Faster, Cheaper

:fire: News :fire:

[Mar 2025] Run and serve Google Gemma 3 using SkyPilot example
[Feb 2025] Prepare and serve Retrieval Augmented Generation (RAG) with DeepSeek-R1: blog post, example
[Feb 2025] Run and serve DeepSeek-R1 671B using SkyPilot and SGLang with high throughput: example
[Feb 2025] Prepare and serve large-scale image search with vector databases: blog post, example
[Jan 2025] Launch and serve distilled models from DeepSeek-R1 and Janus on Kubernetes or any cloud: R1 example and Janus example
[Oct 2024] :tada: SkyPilot crossed 1M+ downloads :tada:: Thank you to our community! Twitter/X
[Sep 2024] Point, launch and serve Llama 3.2 on Kubernetes or any cloud: example
[Sep 2024] Run and deploy Pixtral, the first open-source multimodal model from Mistral AI.
[Jun 2024] Reproduce GPT with llm.c on any cloud: guide
[Apr 2024] Serve Qwen-110B on your infra: example
[Apr 2024] Host Ollama on the cloud to deploy LLMs on CPUs and GPUs: example

LLM Finetuning Cookbooks: Finetuning Llama 2 / Llama 3.1 in your own cloud environment, privately: Llama 2 example and blog; Llama 3.1 example and blog

SkyPilot is an open-source framework for running AI and batch workloads on any infra.

SkyPilot is easy to use for AI users:

Quickly spin up compute on your own infra
Environment and job as code — simple and portable
Easy job management: queue, run, and auto-recover many jobs

SkyPilot unifies multiple clusters, clouds, and hardware:

One interface to use reserved GPUs, Kubernetes clusters, or 16+ clouds
Flexible provisioning of GPUs, TPUs, CPUs, with auto-retry
Team deployment and resource sharing

SkyPilot cuts your cloud costs & maximizes GPU availability:

Autostop: automatic cleanup of idle resources
Spot instance support: 3-6x cost savings, with preemption auto-recovery
Intelligent scheduling: automatically run on the cheapest & most available infra

SkyPilot supports your existing GPU, TPU, and CPU workloads, with no code changes.

Install with pip:

1
2

# Choose your clouds:
pip install -U "skypilot[kubernetes,aws,gcp,azure,oci,lambda,runpod,fluidstack,paperspace,cudo,ibm,scp,nebius]"

To get the latest features and fixes, use the nightly build or install from source:

1
2

# Choose your clouds:
pip install "skypilot-nightly[kubernetes,aws,gcp,azure,oci,lambda,runpod,fluidstack,paperspace,cudo,ibm,scp,nebius]"

Current supported infra: Kubernetes, AWS, GCP, Azure, OCI, Lambda Cloud, Fluidstack, RunPod, Cudo, Digital Ocean, Paperspace, Cloudflare, Samsung, IBM, Vast.ai, VMware vSphere, Nebius.

Getting started

You can find our documentation here.

SkyPilot in 1 minute

A SkyPilot task specifies: resource requirements, data to be synced, setup commands, and the task commands.

Once written in this unified interface (YAML or Python API), the task can be launched on any available cloud. This avoids vendor lock-in, and allows easily moving jobs to a different provider.

Paste the following into a file my_task.yaml:

resources:
  accelerators: A100:8  # 8x NVIDIA A100 GPU

num_nodes: 1  # Number of VMs to launch

# Working directory (optional) containing the project codebase.
# Its contents are synced to ~/sky_workdir/ on the cluster.
workdir: ~/torch_examples

# Commands to be run before executing the job.
# Typical use: pip install -r requirements.txt, git clone, etc.
setup: |
  pip install "torch<2.2" torchvision --index-url https://download.pytorch.org/whl/cu121

# Commands to run as a job.
# Typical use: launch the main program.
run: |
  cd mnist
  python main.py --epochs 1

Prepare the workdir by cloning:

`1`	`git clone https://github.com/pytorch/examples.git ~/torch_examples`

Launch with sky launch (note: access to GPU instances is needed for this example):

`1`	`sky launch my_task.yaml`

SkyPilot then performs the heavy-lifting for you, including:

Find the lowest priced VM instance type across different clouds
Provision the VM, with auto-failover if the cloud returned capacity errors
Sync the local workdir to the VM
Run the task’s setup commands to prepare the VM for running the task
Run the task’s run commands

See Quickstart to get started with SkyPilot.

Runnable examples

See SkyPilot examples that cover: development, training, serving, LLM models, AI apps, and common frameworks.

Latest featured examples:

Task	Examples
Training	PyTorch, DeepSpeed, Finetune Llama 3, NeMo, Ray, Unsloth, Jax/TPU
Serving	vLLM, SGLang, Ollama
Models	DeepSeek-R1, Llama 3, CodeLlama, Qwen, Mixtral
AI apps	RAG, vector databases (ChromaDB, CLIP)
Common frameworks	Airflow, Jupyter

Source files and more examples can be found in llm/ and examples/.

More information

To learn more, see SkyPilot Overview, SkyPilot docs, and SkyPilot blog.

Case studies and integrations: Community Spotlights

Follow updates:

Read the research:

SkyPilot paper and talk (NSDI 2023)
Sky Computing whitepaper
Sky Computing vision paper (HotOS 2021)
SkyServe: AI serving across regions and clouds (EuroSys 2025)
Managed jobs spot instance policy (NSDI 2024)

SkyPilot was initially started at the Sky Computing Lab at UC Berkeley and has since gained many industry contributors. To read about the project’s origin and vision, see Concept: Sky Computing.

Questions and feedback

We are excited to hear your feedback:

For issues and feature requests, please open a GitHub issue.
For questions, please use GitHub Discussions.

For general discussions, join us on the SkyPilot Slack.

Contributing

We welcome all contributions to the project! See CONTRIBUTING for how to get involved.

tensorflow

Fri, 25 Apr 2025 15:28:35 +0800

tensorflow/tensorflow

`Documentation`

TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries, and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML-powered applications.

TensorFlow was originally developed by researchers and engineers working within the Machine Intelligence team at Google Brain to conduct research in machine learning and neural networks. However, the framework is versatile enough to be used in other areas as well.

TensorFlow provides stable Python and C++ APIs, as well as a non-guaranteed backward compatible API for other languages.

Keep up-to-date with release announcements and security updates by subscribing to announce@tensorflow.org. See all the mailing lists.

Install

See the TensorFlow install guide for the pip package, to enable GPU support, use a Docker container, and build from source.

To install the current release, which includes support for CUDA-enabled GPU cards (Ubuntu and Windows):

`1`	`$ pip install tensorflow`

Other devices (DirectX and MacOS-metal) are supported using Device plugins.

A smaller CPU-only package is also available:

`1`	`$ pip install tensorflow-cpu`

To update TensorFlow to the latest version, add --upgrade flag to the above commands.

Nightly binaries are available for testing using the tf-nightly and tf-nightly-cpu packages on PyPI.

Try your first TensorFlow program

`1`	`$ python`

>>> import tensorflow as tf
>>> tf.add(1, 2).numpy()
3
>>> hello = tf.constant('Hello, TensorFlow!')
>>> hello.numpy()
b'Hello, TensorFlow!'

For more examples, see the TensorFlow tutorials.

Contribution guidelines

If you want to contribute to TensorFlow, be sure to review the contribution guidelines. This project adheres to TensorFlow’s code of conduct. By participating, you are expected to uphold this code.

We use GitHub issues for tracking requests and bugs, please see TensorFlow Forum for general questions and discussion, and please direct specific questions to Stack Overflow.

The TensorFlow project strives to abide by generally accepted best practices in open-source software development.

Patching guidelines

Follow these steps to patch a specific version of TensorFlow, for example, to apply fixes to bugs or security vulnerabilities:

Clone the TensorFlow repo and switch to the corresponding branch for your desired TensorFlow version, for example, branch r2.8 for version 2.8.
Apply (that is, cherry-pick) the desired changes and resolve any code conflicts.
Run TensorFlow tests and ensure they pass.
Build the TensorFlow pip package from source.

Continuous build status

You can find more community-supported platforms and configurations in the TensorFlow SIG Build community builds table.

Official Builds

Build Type	Status	Artifacts
Linux CPU		PyPI
Linux GPU		PyPI
Linux XLA		TBA
macOS		PyPI
Windows CPU		PyPI
Windows GPU		PyPI
Android		Download
Raspberry Pi 0 and 1		Py3
Raspberry Pi 2 and 3		Py3
Libtensorflow MacOS CPU	Status Temporarily Unavailable	Nightly Binary Official GCS
Libtensorflow Linux CPU	Status Temporarily Unavailable	Nightly Binary Official GCS
Libtensorflow Linux GPU	Status Temporarily Unavailable	Nightly Binary Official GCS
Libtensorflow Windows CPU	Status Temporarily Unavailable	Nightly Binary Official GCS
Libtensorflow Windows GPU	Status Temporarily Unavailable	Nightly Binary Official GCS

Resources

Learn more about the TensorFlow community and how to contribute.

Courses

License

Apache License 2.0

cuda-python

Fri, 11 Apr 2025 15:27:47 +0800

NVIDIA/cuda-python

cuda-python

CUDA Python is the home for accessing NVIDIA’s CUDA platform from Python. It consists of multiple components:

cuda.core: Pythonic access to CUDA Runtime and other core functionalities
cuda.bindings: Low-level Python bindings to CUDA C APIs
cuda.cooperative: A Python package providing CCCL’s reusable block-wide and warp-wide device primitives for use within Numba CUDA kernels
cuda.parallel: A Python package for easy access to CCCL’s highly efficient and customizable parallel algorithms, like sort, scan, reduce, transform, etc, that are callable on the host
numba.cuda: Numba’s target for CUDA GPU programming by directly compiling a restricted subset of Python code into CUDA kernels and device functions following the CUDA execution model.

For access to NVIDIA CPU & GPU Math Libraries, please refer to nvmath-python.

CUDA Python is currently undergoing an overhaul to improve existing and bring up new components. All of the previously available functionalities from the cuda-python package will continue to be available, please refer to the cuda.bindings documentation for installation guide and further detail.

cuda-python as a metapackage

cuda-python is being re-structured to become a metapackage that contains a collection of subpackages. Each subpackage is versioned independently, allowing installation of each component as needed.

Subpackage: `cuda.core`

The cuda.core package offers idiomatic, pythonic access to CUDA Runtime and other functionalities.

The goals are to

Provide idiomatic (“pythonic”) access to CUDA Driver, Runtime, and JIT compiler toolchain
Focus on developer productivity by ensuring end-to-end CUDA development can be performed quickly and entirely in Python
Avoid homegrown Python abstractions for CUDA for new Python GPU libraries starting from scratch
Ease developer burden of maintaining and catching up with latest CUDA features
Flatten the learning curve for current and future generations of CUDA developers

Subpackage: `cuda.bindings`

The cuda.bindings package is a standard set of low-level interfaces, providing full coverage of and access to the CUDA host APIs from Python.

The list of available interfaces are:

CUDA Driver
CUDA Runtime
NVRTC
nvJitLink
NVVM

GPU on Producthunt daily

Sunshine

Sunshine

Self-hosted game stream host for Moonlight.

ℹ️ About

🖥️ System Requirements

❓ Support

💲 Sponsors and Supporters

👥 Contributors

GitHub

CrowdIn

LMCache

Summary

Features

Installation

Getting started

Documentation

Examples

Interested in Connecting?

Community meeting

Contributing

Citation

Socials

License

hashcat

hashcat

License

Installation

Usage/Help

Building

Contributing

Happy Cracking!

ZLUDA

ZLUDA

Usage

Windows

Linux

MacOS

Building

Dependencies

Build steps

Contributing

Getting started

License

rl-swarm

RL Swarm

Requirements

⚠️ Please read before continuing ⚠️

Instructions

Run the Swarm

1. Clone this repo

2. Install Docker

3. Start the Swarm

CPU support

GPU support

Docker compose issue

Experimental (advanced) mode

Login

Huggingface

Initial peering and training

Identity management

Introduction

What this means

Troubleshooting

tinygrad

Homepage | Documentation | Discord

Features

LLaMA and Stable Diffusion

Laziness

Neural networks

Accelerators

Installation

From source

Direct (master)

Documentation

Quick example comparing to PyTorch

Contributing

Running tests

Process replay tests

skypilot

Subpackage: `cuda.core`

Subpackage: `cuda.bindings`