Lightweight on Producthunt daily

nano-vllm

Wed, 05 Nov 2025 15:29:19 +0800

GeeeekExplorer/nano-vllm

Nano-vLLM

A lightweight vLLM implementation built from scratch.

Key Features

🚀 Fast offline inference - Comparable inference speeds to vLLM
📖 Readable codebase - Clean implementation in ~ 1,200 lines of Python code
⚡ Optimization Suite - Prefix caching, Tensor Parallelism, Torch compilation, CUDA graph, etc.

Installation

`1`	`pip install git+https://github.com/GeeeekExplorer/nano-vllm.git`

Model Download

To download the model weights manually, use the following command:

1
2
3

huggingface-cli download --resume-download Qwen/Qwen3-0.6B \
  --local-dir ~/huggingface/Qwen3-0.6B/ \
  --local-dir-use-symlinks False

Quick Start

See example.py for usage. The API mirrors vLLM’s interface with minor differences in the LLM.generate method:

from nanovllm import LLM, SamplingParams
llm = LLM("/YOUR/MODEL/PATH", enforce_eager=True, tensor_parallel_size=1)
sampling_params = SamplingParams(temperature=0.6, max_tokens=256)
prompts = ["Hello, Nano-vLLM."]
outputs = llm.generate(prompts, sampling_params)
outputs[0]["text"]

Benchmark

See bench.py for benchmark.

Test Configuration:

Hardware: RTX 4070 Laptop (8GB)
Model: Qwen3-0.6B
Total Requests: 256 sequences
Input Length: Randomly sampled between 100–1024 tokens
Output Length: Randomly sampled between 100–1024 tokens

Performance Results:

Inference Engine	Output Tokens	Time (s)	Throughput (tokens/s)
vLLM	133,966	98.37	1361.84
Nano-vLLM	133,966	93.41	1434.13

Star History

Pake

Wed, 27 Aug 2025 15:28:19 +0800

tw93/Pake

English | 简体中文 | 日本語

Pake

Turn any webpage into a desktop app with Rust with ease.

Pake supports Mac, Windows, and Linux. Check out README for Popular Packages, Command-Line Packaging, and Customized Development information. Feel free to share your suggestions in Discussions.

Features

🎐 Nearly 20 times smaller than an Electron package (around 5M!)
🚀 With Rust Tauri, Pake is much more lightweight and faster than JS-based frameworks.
📦 Battery-included package — shortcut pass-through, immersive windows, and minimalist customization.
👻 Pake is just a simple tool — replace the old bundle approach with Tauri (though PWA is good enough).

Popular Packages

WeRead Mac Windows Linux	Twitter Mac Windows Linux

Grok Mac Windows Linux	DeepSeek Mac Windows Linux

ChatGPT Mac Windows Linux	Gemini Mac Windows Linux

YouTube Music Mac Windows Linux	YouTube Mac Windows Linux

LiZhi Mac Windows Linux	ProgramMusic Mac Windows Linux

Excalidraw Mac Windows Linux	XiaoHongShu Mac Windows Linux

🏂 You can download more applications from Releases. Click here to expand the shortcuts reference!

Mac	Windows/Linux	Function
`⌘` + `[`	`Ctrl` + `←`	Return to the previous page
`⌘` + `]`	`Ctrl` + `→`	Go to the next page
`⌘` + `↑`	`Ctrl` + `↑`	Auto scroll to top of page
`⌘` + `↓`	`Ctrl` + `↓`	Auto scroll to bottom of page
`⌘` + `r`	`Ctrl` + `r`	Refresh Page
`⌘` + `w`	`Ctrl` + `w`	Hide window, not quite
`⌘` + `-`	`Ctrl` + `-`	Zoom out the page
`⌘` + `+`	`Ctrl` + `+`	Zoom in the page
`⌘` + `=`	`Ctrl` + `=`	Zoom in the Page
`⌘` + `0`	`Ctrl` + `0`	Reset the page zoom

In addition, double-click the title bar to switch to full-screen mode. For Mac users, you can also use the gesture to go to the previous or next page and drag the title bar to move the window.

Before starting

For beginners: Play with Popular Packages to find out Pake’s capabilities, or try to pack your application with GitHub Actions. Don’t hesitate to reach for assistance at Discussion!
For developers: “Command-Line Packaging” supports macOS fully. For Windows/Linux users, it requires some tinkering. Configure your environment before getting started.
For hackers: For people who are good at both front-end development and Rust, how about customizing your apps’ function more with the following Customized Development?

Command-Line Packaging

Pake provides a command line tool, making the flow of package customization quicker and easier. See documentation for more information.

# Install with npm
npm install -g pake-cli

# Command usage
pake url [OPTIONS]...

# Feel free to play with Pake! It might take a while to prepare the environment the first time you launch Pake.
pake https://weekly.tw93.fun --name Weekly --hide-title-bar

If you are new to the command line, you can compile packages online with GitHub Actions. See the Tutorial for more information.

Development

Prepare your environment before starting. Make sure you have Rust >=1.89 and Node >=22 (e.g., 22.11.0) installed on your computer. Note: Older versions (Rust ≥1.78, Node ≥16) may also work but latest stable versions are recommended. For installation guidance, see Tauri documentation.

If you are unfamiliar with these, it is better to try out the above tool to pack with one click.

# Install Dependencies
npm i

# Local development [Right-click to open debug mode.]
npm run dev

# Pack application
npm run build

Advanced Usage

You can refer to the codebase structure before working on Pake, which will help you much in development.
Modify the url and productName fields in the pake.json file under the src-tauri directory, the “domain” field in the tauri.config.json file needs to be modified synchronously, as well as the icon and identifier fields in the tauri.xxx.conf.json file. You can select an icon from the icons directory or download one from macOSicons to match your product needs.
For configurations on window properties, you can modify the pake.json file to change the value of width, height, fullscreen (or not), resizable (or not), hide_on_close (hide window on close instead of exiting) of the windows property. To adapt to the immersive header on Mac, change hideTitleBar to true, look for the Header element, and add the padding-top property.
For advanced usages such as style rewriting, advertisement removal, JS injection, container message communication, and user-defined shortcut keys, see Advanced Usage of Pake.

Developers

Pake’s development can not be without these Hackers. They contributed a lot of capabilities for Pake. Also, welcome to follow them! ❤️

_Tw93	_Tlntin	_Santree	_Pan93412	_{Данил Бизимов}	_Volare	_{Bryan Lee}
_Essesoul	_{Jerry Zhou}	_Aiello	_Horus	_{Pake Actions}	_GoodbyeNJN	_Kittizz
_{Matt Bajorek}	_{Srinivas Vaddi}	_Steam	_Qitianjia	_{Yi Xin}	_{Yue Yang}	_Kieran
_孟世博	_2nthony	_Null	_{Abu Taher Siddik}	_{An Li}	_{Ayaka Neko}	_{Dengju Deng}
_Fabien	_Fechin	_Imgbot	_{Jiaqi Gu}	_Milo	_{Po Chen}	_{Xie Ruiqi}
_Null	_Null	_Hyzhao	_Null	_Liudonghua	_Liusishan	_Ranger
_贺天卓

Frequently Asked Questions

Right-clicking on an image element in the page to open the menu and select download image or other events does not work (common in MacOS systems). This issue is due to the MacOS built-in webview not supporting this feature.

Support

I have two cats, TangYuan and Coke. If you think Pake delights your life, you can feed them some canned food 🥩.
If you like Pake, you can star it on GitHub. Also, welcome to recommend Pake to your friends.
You can follow my Twitter to get the latest news of Pake or join our Telegram chat group.
I hope that you enjoy playing with it. Let us know if you find a website that would be great for a Mac App!

WasmEdge

Fri, 18 Jul 2025 15:35:26 +0800

WasmEdge/WasmEdge

中文 | 正體中文 | 日本語で読む

🤩 WasmEdge is the easiest and fastest way to run LLMs on your own devices. 🤩

WasmEdge is a lightweight, high-performance, and extensible WebAssembly runtime. It is the fastest Wasm VM. WasmEdge is an official sandbox project hosted by the CNCF. LlamaEdge is an application framework built on top of WasmEdge to run GenAI models (e.g., LLM, speech-to-text, text-to-image, and TTS) across GPUs on servers, personal computers, and edge devices. Additional use cases include microservices on the edge cloud, serverless SaaS APIs, embedded functions, smart contracts, and smart devices.

Quick start guides

🚀 Install WasmEdge
👷🏻‍♂️ Build and contribute to WasmEdge
⌨️ Run a standalone Wasm program or a JavaScript program from CLI or Docker
🤖 Chat with an open source LLM via LlamaEdge
🔌 Embed a Wasm function in your Go, Rust, or C app
🛠 Manage and orchestrate Wasm runtimes using Kubernetes, data streaming frameworks, and blockchains
📚 Check out our official documentation

Introduction

The WasmEdge Runtime provides a well-defined execution sandbox for its contained WebAssembly bytecode program. The runtime offers isolation and protection for operating system resources (e.g., file system, sockets, environment variables, processes) and memory space. The most important use case for WasmEdge is to safely execute user-defined or community-contributed code as plug-ins in a software product (e.g., SaaS, software-defined vehicles, edge nodes, or even blockchain nodes). It enables third-party developers, vendors, suppliers, and community members to extend and customize the software product. Learn more here

Performance

A Lightweight Design for High-performance Serverless Computing, published on IEEE Software, Jan 2021. https://arxiv.org/abs/2010.07115
Performance Analysis for Arm vs. x86 CPUs in the Cloud, published on infoQ.com, Jan 2021. https://www.infoq.com/articles/arm-vs-x86-cloud-performance/
WasmEdge is the fastest WebAssembly Runtime in Suborbital Reactr test suite, Dec 2021

Features

WasmEdge can run standard WebAssembly bytecode programs compiled from C/C++, Rust, Swift, AssemblyScript, or Kotlin source code. It runs JavaScript, including 3rd party ES6, CJS, and NPM modules, in a secure, fast, lightweight, portable, and containerized sandbox. It also supports mixing of those languages (e.g., to use Rust to implement a JavaScript API), the Fetch API, and Server-side Rendering (SSR) functions on edge servers.

WasmEdge supports all standard WebAssembly features and many proposed extensions. It also supports a number of extensions tailored for cloud-native and edge computing uses (e.g., the WasmEdge network sockets,Postgres and MySQL-based database driver, and the WasmEdge AI extension).

Learn more about technical highlights of WasmEdge.

Integrations and management

WasmEdge and its contained wasm program can be started from the CLI as a new process, or from an existing process. If started from an existing process (e.g., from a running Go or Rust program), WasmEdge will simply run inside the process as a function. Currently, WasmEdge is not yet thread-safe. In order to use WasmEdge in your own application or cloud-native frameworks, please refer to the guides below.

Community

Contributing

We welcome contributions from the community! Please check out our:

Contributing Guide for how to get started
Governance documentation for project decision-making processes
Code of Conduct for community standards

Want to become a maintainer? See our Contributor Ladder.

Roadmap

Check out our project roadmap to see the upcoming features and plans for WasmEdge.

Contact

If you have any questions, feel free to open a GitHub issue on a related project or to join the following channels:

Mailing list: Send an email to WasmEdge@googlegroups.com
Discord: Join the WasmEdge Discord server!
Slack: Join the #WasmEdge channel on the CNCF Slack
X (formerly Twitter): Follow @realwasmedge on X

Adopters

Check out our list of Adopters who are using WasmEdge in their projects.

Community Meeting

We host a monthly community meeting to showcase new features, demo new use cases, and a Q&A part. Everyone is welcome!

Time: The first Tuesday of each month at 11PM Hong Kong Time/ 7AM PST.

Public meeting agenda/notes | Zoom link

Lightweight on Producthunt daily

nano-vllm

GeeeekExplorer/nano-vllm

Nano-vLLM

Key Features

Installation

Model Download

Quick Start

Benchmark

Star History

Pake

tw93/Pake

English | 简体中文 | 日本語

Pake

Features

Popular Packages

Before starting

Command-Line Packaging

Development

Advanced Usage

Developers

Frequently Asked Questions

Support

WasmEdge

WasmEdge/WasmEdge

🤩 WasmEdge is the easiest and fastest way to run LLMs on your own devices. 🤩

Quick start guides

Introduction

Performance

Features

Integrations and management

Community

Contributing

Roadmap

Contact

Adopters

Community Meeting

License