Generative AI on Producthunt daily

gallery

Mon, 06 Apr 2026 16:25:40 +0800

google-ai-edge/gallery

Google AI Edge Gallery ✨

Explore, Experience, and Evaluate the Future of On-Device Generative AI with Google AI Edge.

AI Edge Gallery is the premier destination for running the world’s most powerful open-source Large Language Models (LLMs) on your mobile device. Experience high-performance Generative AI directly on your hardware—fully offline, private, and lightning-fast.

Now Featuring: Gemma 4

The latest version brings official support for the newly released Gemma 4 family. As the centerpiece of this release, Gemma 4 allows you to test the cutting edge of on-device AI. Experience advanced reasoning, logic, and creative capabilities without ever sending your data to a server.

Install the app today from Google Play	Install the app today from App Store

For users without Google Play access, install the apk from the latest release

App Preview

✨ Core Features

Agent Skills: Transform your LLM from a conversationalist into a proactive assistant. Use the Agent Skills tile to augment model capabilities with tools like Wikipedia for fact-grounding, interactive maps, and rich visual summary cards. You can even load modular skills from a URL or browse community contributions on GitHub Discussions.
AI Chat with Thinking Mode: Engage in fluid, multi-turn conversations and toggle the new Thinking Mode to peek “under the hood.” This feature allows you to see the model’s step-by-step reasoning process, which is perfect for understanding complex problem-solving. Note: Thinking Mode currently works with supported models, starting with the Gemma 4 family.
Ask Image: Use multimodal power to identify objects, solve visual puzzles, or get detailed descriptions using your device’s camera or photo gallery.
Audio Scribe: Transcribe and translate voice recordings into text in real-time using high-efficiency on-device language models.
Prompt Lab: A dedicated workspace to test different prompts and single-turn use cases with granular control over model parameters like temperature and top-k.
Mobile Actions: Unlock offline device controls and automated tasks powered entirely by a finetune of FuntionGemma 270m.
Tiny Garden: A fun, experimental mini-game that uses natural language to plant and harvest a virtual garden using a finetune of FunctionGemma 270m.
Model Management & Benchmark: Gallery is a flexible sandbox for a wide variety of open-source models. Easily download models from the list or load your own custom models. Manage your model library effortlessly and run benchmark tests to understand exactly how each model performs on your specific hardware.
100% On-Device Privacy: All model inferences happen directly on your device hardware. No internet is required, ensuring total privacy for your prompts, images, and sensitive data.

🏁 Get Started in Minutes!

Check OS Requirement: Android 12 and up, and iOS 17 and up.
Download the App:
- Install the app from Google Play or App Store.
- For users without Google Play access: install the apk from the latest release
Install & Explore: For detailed installation instructions (including for corporate devices) and a full user guide, head over to our Project Wiki!

🛠️ Technology Highlights

Google AI Edge: Core APIs and tools for on-device ML.
LiteRT: Lightweight runtime for optimized model execution.
Hugging Face Integration: For model discovery and download.

⌨️ Development

Check out the development notes for instructions about how to build the app locally.

🤝 Feedback

This is an experimental Beta release, and your input is crucial!

🐞 Found a bug? Report it here!
💡 Have an idea? Suggest a feature!

📄 License

Licensed under the Apache License, Version 2.0. See the LICENSE file for details.

🔗 Useful Links

generative-ai-for-beginners

Tue, 09 Sep 2025 15:28:49 +0800

microsoft/generative-ai-for-beginners

21 Lessons teaching everything you need to know to start building Generative AI applications

🌐 Multi-Language Support

Supported via GitHub Action (Automated & Always Up-to-Date)

Generative AI for Beginners (Version 3) - A Course

Learn the fundamentals of building Generative AI applications with our 21-lesson comprehensive course by Microsoft Cloud Advocates.

🌱 Getting Started

This course has 21 lessons. Each lesson covers its own topic so start wherever you like!

Lessons are labeled either “Learn” lessons explaining a Generative AI concept or “Build” lessons that explain a concept and code examples in both Python and TypeScript when possible.

For .NET Developers checkout Generative AI for Beginners (.NET Edition)!

Each lesson also includes a “Keep Learning” section with additional learning tools.

What You Need

To run the code of this course, you can use either:

Azure OpenAI Service - Lessons: “aoai-assignment”
GitHub Marketplace Model Catalog - Lessons: “githubmodels”
OpenAI API - Lessons: “oai-assignment”
Basic knowledge of Python or TypeScript is helpful - *For absolute beginners check out these Python and TypeScript courses
A GitHub account to fork this entire repo to your own GitHub account

We have created a Course Setup lesson to help you with setting up your development environment.

Don’t forget to star (🌟) this repo to find it easier later.

🧠 Ready to Deploy?

If you are looking for more advanced code samples, check out our collection of Generative AI Code Samples in both Python and TypeScript.

🗣️ Meet Other Learners, Get Support

Join our official Azure AI Foundry Discord server to meet and network with other learners taking this course and get support.

Ask questions or share product feedback in our Azure AI Foundry Developer Forum on Github.

🚀 Building a Startup?

Visit Microsoft for Startups to find out how to get started building with Azure credits today.

🙏 Want to help?

Do you have suggestions or found spelling or code errors? Raise an issue or Create a pull request

📂 Each lesson includes:

A short video introduction to the topic
A written lesson located in the README
Python and TypeScript code samples supporting Azure OpenAI and OpenAI API
Links to extra resources to continue your learning

🗃️ Lessons

#	Lesson Link	Description	Video	Extra Learning
00	Course Setup	Learn: How to Setup Your Development Environment	Video Coming Soon	Learn More
01	Introduction to Generative AI and LLMs	Learn: Understanding what Generative AI is and how Large Language Models (LLMs) work.	Video	Learn More
02	Exploring and comparing different LLMs	Learn: How to select the right model for your use case	Video	Learn More
03	Using Generative AI Responsibly	Learn: How to build Generative AI Applications responsibly	Video	Learn More
04	Understanding Prompt Engineering Fundamentals	Learn: Hands-on Prompt Engineering Best Practices	Video	Learn More
05	Creating Advanced Prompts	Learn: How to apply prompt engineering techniques that improve the outcome of your prompts.	Video	Learn More
06	Building Text Generation Applications	Build: A text generation app using Azure OpenAI / OpenAI API	Video	Learn More
07	Building Chat Applications	Build: Techniques for efficiently building and integrating chat applications.	Video	Learn More
08	Building Search Apps Vector Databases	Build: A search application that uses Embeddings to search for data.	Video	Learn More
09	Building Image Generation Applications	Build: An image generation application	Video	Learn More
10	Building Low Code AI Applications	Build: A Generative AI application using Low Code tools	Video	Learn More
11	Integrating External Applications with Function Calling	Build: What is function calling and its use cases for applications	Video	Learn More
12	Designing UX for AI Applications	Learn: How to apply UX design principles when developing Generative AI Applications	Video	Learn More
13	Securing Your Generative AI Applications	Learn: The threats and risks to AI systems and methods to secure these systems.	Video	Learn More
14	The Generative AI Application Lifecycle	Learn: The tools and metrics to manage the LLM Lifecycle and LLMOps	Video	Learn More
15	Retrieval Augmented Generation (RAG) and Vector Databases	Build: An application using a RAG Framework to retrieve embeddings from a Vector Databases	Video	Learn More
16	Open Source Models and Hugging Face	Build: An application using open source models available on Hugging Face	Video	Learn More
17	AI Agents	Build: An application using an AI Agent Framework	Video	Learn More
18	Fine-Tuning LLMs	Learn: The what, why and how of fine-tuning LLMs	Video	Learn More
19	Building with SLMs	Learn: The benefits of building with Small Language Models	Video Coming Soon	Learn More
20	Building with Mistral Models	Learn: The features and differences of the Mistral Family Models	Video Coming Soon	Learn More
21	Building with Meta Models	Learn: The features and differences of the Meta Family Models	Video Coming Soon	Learn More

🌟 Special thanks

Special thanks to John Aziz for creating all of the GitHub Actions and workflows

Bernhard Merkle for making key contributions to each lesson to improve the learner and code experience.

🎒 Other Courses

Our team produces other courses! Check out:

awesome-generative-ai

Sat, 26 Jul 2025 15:30:16 +0800

steven2358/awesome-generative-ai

Awesome Generative AI

A curated list of modern Generative Artificial Intelligence projects and services.

Generative Artificial Intelligence is a technology that creates original content such as images, sounds, and texts by using machine learning algorithms that are trained on large amounts of data. Unlike other forms of AI, it is capable of creating unique and previously unseen outputs such as photorealistic images, digital art, music, and writing. These outputs often have their own unique style and can even be hard to distinguish from human-created works. Generative AI has a wide range of applications in fields such as of art, entertainment, marketing, academia, and computer science.

Contributions to this list are welcome. Before submitting your suggestions, please review the Contribution Guidelines to ensure your entries meet the criteria. Add links through pull requests or create an issue to start a discussion. More projects can be found in the Discoveries List, where we showcase a wide range of up-and-coming Generative AI projects.

Text

Models

OpenAI API - OpenAI’s API provides access to GPT-3 and GPT-4 models, which performs a wide variety of natural language tasks, and Codex, which translates natural language to code.
Gopher - Gopher by DeepMind is a 280 billion parameter language model.
OPT - Open Pretrained Transformers (OPT) by Facebook is a suite of decoder-only pre-trained transformers. Announcement. OPT-175B text generation hosted by Alpa.
Bloom - BLOOM by Hugging Face is a model similar to GPT-3 that has been trained on 46 different languages and 13 programming languages. #opensource
Llama - Meta’s open source large language model. #opensource
Claude - Talk to Claude, an AI assistant from Anthropic.
Vicuna-13B - An open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT. #opensource
Mistral - Cutting-edge open-weight LLMs by Mistral AI. #opensource
Grok - An LLM by xAI with open source and open weights. #opensource
Qwen - A series of LLMs independently developed by Alibaba Cloud. #opensource

Chatbots

ChatGPT - ChatGPT by OpenAI is a large language model that interacts in a conversational way.
Copilot - An everyday AI companion by Microsoft.
Gemini - A family of multimodal large language model developed by Google Deepmind.
Meta AI - Meta AI assistant to get things done, create AI-generated images, get answers. Built on Llama LLM.
DeepSeek - Cutting-edge LLMs for enterprise, consumer, and scientific applications. #opensource
Character.AI - Character.AI lets you create characters and chat to them.
Pi - A personalized AI platform available as a digital assistant.
Qwen - Qwen chatbot with image generation, document processing, web search integration, video understanding, etc.
Le Chat - Chat with Mistral AI’s cutting-edge language models.

Custom interfaces

LibreChat - LibreChat is a free and open-source chat interface for assistant AIs. #opensource.
Chatbot UI - An open source ChatGPT UI. #opensource.

Search engines

Perplexity AI - AI powered search tools.
Metaphor - Language model powered search.
Phind - AI-based search engine.
You.com - A search engine built on AI that provides users with a customized search experience while keeping their data 100% private.
Komo - An AI-powered search engine.

Local search engines

privateGPT - Ask questions to your documents without an internet connection, using the power of LLMs.
quivr - Dump all your files and chat with it using your generative AI second brain using LLMs & embeddings.

Writing assistants

Jasper - Create content faster with artificial intelligence.
Compose AI - Compose AI is a free Chrome extension that cuts your writing time by 40% with AI-powered autocompletion.
Rytr - Rytr is an AI writing assistant that helps you create high-quality content.
wordtune - Personal writing assistant.
HyperWrite - HyperWrite helps you write with confidence and get your work done faster from idea to final draft.
Moonbeam - Better blogs in a fraction of the time.
copy.ai - Write better marketing copy and content with AI.
ChatSonic - An AI-powered assistant that enables text and image creation.
Anyword - Anyword’s AI writing assistant generates effective copy for anyone.
Hypotenuse AI - Turn a few keywords into original, insightful articles, product descriptions and social media copy.
Lavender - Lavender email assistant helps you get more replies in less time.
Lex - A word processor with artificial intelligence baked in, so you can write faster.
Jenni - Jenni is the ultimate writing assistant that saves you hours of ideation and writing time.
LAIKA - LAIKA trains an artificial intelligence on your own writing to create a personalised creative partner-in-crime.
QuillBot - AI-powered paraphrasing tool.
Postwise - Write tweets, schedule posts and grow your following using AI.
Copysmith - AI content creation solution for Enterprise & eCommerce.

ChatGPT extensions

WebChatGPT - Augment your ChatGPT prompts with relevant results from the web.
GPT for Sheets and Docs - ChatGPT extension for Google Sheets and Google Docs.
YouTube Summary with ChatGPT - Use ChatGPT to summarize YouTube videos.
ChatGPT Prompt Genius - Discover, share, import, and use the best prompts for ChatGPT & save your chat history locally.
ChatGPT for Search Engines - Display ChatGPT response alongside Google, Bing, and DuckDuckGo search results.
ShareGPT - Share your ChatGPT conversations and explore conversations shared by others.
Merlin - ChatGPT Plus extension on all websites.
ChatGPT Writer - Generate entire emails and messages using ChatGPT AI.
ChatGPT for Jupyter - Add various helper functions in Jupyter Notebooks and Jupyter Lab, powered by ChatGPT.
editGPT - Easily proofread, edit, and track changes to your content in chatGPT.
Forefront - A Better ChatGPT Experience.

Productivity

ChatPDF - Chat with any PDF.
Mem - Mem is the world’s first AI-powered workspace that’s personalized to you. Amplify your creativity, automate the mundane, and stay organized automatically.
Taskade - Outline tasks, notes, generated structured lists and mind maps with Taskade AI.
Notion AI - Write better, more efficient notes and docs.
Nekton AI - Automate your workflows with AI. Describe your workflows step by step in plain language.
Rewind - Rewind is a personalized AI powered by everything you’ve seen, said, or heard.
NotebookLM - A research and note-taking online tool to interact with documents, powered by Google Gemini.

Meeting assistants

Otter.ai - A meeting assistant that records audio, writes notes, automatically captures slides, and generates summaries.
Cogram - Cogram takes automatic notes in virtual meetings and identifies action items.
Sybill - Sybill generates summaries of sales calls, including next steps, pain points and areas of interest, by combining transcript and emotion-based insights.
Loopin AI - Loopin is a collaborative meeting workspace that not only enables you to record, transcribe & summaries meetings using AI, but also enables you to auto-organise meeting notes on top of your calendar.
Fireflies.ai - Fireflies.ai helps your team transcribe, summarize, search, and analyze voice conversations.
Read AI - An AI copilot for wherever you work, making your meetings, emails, and messages more productive with summaries, content discovery, and recommendations.
Fireflies.ai - Transcribe, summarize, search, and analyze all your team conversations.

Academia

Elicit - Elicit uses language models to help you automate research workflows, like parts of literature review.
genei - Summarise academic articles in seconds and save 80% on your research times.
Explainpaper - A better way to read academic papers. Upload a paper, highlight confusing text, get an explanation.
Galactica - A large language model for science. Can summarize academic literature, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more. Model API.
Consensus - Consensus is a search engine that uses AI to find answers in scientific research.
Synthical - AI-powered collaborative research environment.
scite - A platform for discovering and evaluating scientific articles.
SciSpace - An AI research assistant for understanding scientific literature.
STORM - An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations. #opensource

Leaderboards

Chatbot Arena - An open platform for crowdsourced AI benchmarking, hosted by researchers at UC Berkeley SkyLab and LMArena.
Artificial Analysis - Artificial Analysis provides objective benchmarks & information to help choose AI models and hosting providers.
imgsys - A generative image model arena by fal.ai.

Other text generators

EmailTriager - Use AI to automatically draft email replies in the background.
AI Poem Generator - AI Poem Generator writes a beautiful rhyming poem for you on any subject, given a text prompt.

Coding

Coding Assistants

GitHub Copilot - GitHub Copilot uses the OpenAI Codex to suggest code and entire functions in real-time, right from your editor.
OpenAI Codex - An AI system by OpenAI that translates natural language to code.
Ghostwriter - An AI-powered pair programmer by replit.
Amazon Q - The AWS generative AI–powered assistant that helps answer questions, write code, and automate tasks.
tabnine - Code faster with whole-line & full-function code completions.
Stenography - Automatic code documentation.
Mintlify - AI powered documentation writer.
Debuild - AI-powered low-code tool for web apps.
AI2sql - With AI2sql, engineers and non-engineers can easily write efficient, error-free SQL queries without knowing SQL.
CodiumAI - With CodiumAI, you get non-trivial tests suggested right inside your IDE, so you stay confident when you push.
PR-Agent - AI-powered tool for automated PR analysis, feedback, suggestions and more.
MutableAI - AI Accelerated Software Development.
TurboPilot - A self-hosted copilot clone which uses the library behind llama.cpp to run the 6 billion parameter Salesforce Codegen model in 4 GB of RAM.
GPT-Code UI - An open source implementation of OpenAI’s ChatGPT Code interpreter. #opensource
MetaGPT - The Multi-Agent Framework: Given one line Requirement, return PRD, Design, Tasks, Repo
Open Interpreter - OpenAI’s Code Interpreter in your terminal, running locally.
Continue - Open-source AI code assistant. Connect any model and any context to create custom autocomplete and chat experiences inside the IDE. #opensource

Developer tools

co:here - Cohere provides access to advanced Large Language Models and NLP tools.
Haystack - A framework for building NLP applications (e.g. agents, semantic search, question-answering) with language models.
LangChain - A framework for developing applications powered by language models.
gpt4all - A chatbot trained on a massive collection of clean assistant data including code, stories and dialogue.
LLM App - Open-source Python library to build real-time LLM-enabled data pipeline.
LMQL - LMQL is a query language for large language models.
LlamaIndex - A data framework for building LLM applications over external data.
Phoenix - Open-source tool for ML observability that runs in your notebook environment, by Arize. Monitor and fine tune LLM, CV and tabular models.
Cursor - Cursor is the IDE of the future, built for pair-programming with Powerful AI.
SymbolicAI - A neuro-symbolic framework for building applications with LLMs at the core.
Vanna.ai - An open-source Python RAG framework for SQL generation and related functionality. #opensource
Portkey - A full-stack LLMOps platform for LLM monitoring, caching, and management.
agenta - An open-source end-to-end LLMOps platform for prompt engineering, evaluation, and deployment. #opensource
Together AI - Train, fine-tune-and run inference on AI models blazing fast, at low cost, and at production scale.
Gitingest - Turn any Git repository into a simple text digest of its codebase so it can be fed into any LLM. #opensource
Repomix - Pack your codebase into AI-friendly formats. #opensource
llama.cpp - Inference of Meta’s LLaMA model (and others) in pure C/C++. #opensource
bitnet.cpp - Official inference framework for 1-bit LLMs, by Microsoft. #opensource
OpenRouter - A unified interface for LLMs. #opensource
Ludwig - A low-code framework for building custom AI models like LLMs and other deep neural networks. #opensource

Playgrounds

OpenAI Playground - Explore resources, tutorials, API docs, and dynamic examples.
Google AI Studio - A web-based tool to prototype with Gemini and experimental models.
GitHub Models - Find and experiment with AI models to develop a generative AI application.

Local LLM Deployment

Ollama - Get up and running with large language models locally.
Open WebUI - An extensible, feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline. #opensource
Jan - Run LLMs like Mistral or Llama2 locally and offline on your computer, or connect to remote AI APIs. #opensource
Msty - A straightworward and powerful interface for local and online AI models.
PyGPT - Personal desktop AI assistant with chat, vision, agents, image generation, tools and commands, voice control and more. #opensource
LLM - A CLI utility and Python library for interacting with Large Language Models, remote and local. #opensource

Agents

Autonomous agents

Auto-GPT - An experimental open-source attempt to make GPT-4 fully autonomous.
babyagi - An AI-powered task management system.
AgentGPT - Assemble, configure, and deploy autonomous AI Agents in your browser.
GPT Engineer - Specify what you want it to build, the AI asks for clarification, and then builds it.
GPT Prompt Engineer - Automated prompt engineering. It generates, tests, and ranks prompts to find the best ones.
MetaGPT - The Multi-Agent Framework: Given one line requirement, return PRD, design, tasks, repo.
AutoGen - AutoGen is a framework that enables the development of LLM applications using multiple agents that can converse with each other to solve tasks.
GPT Pilot - Dev tool that writes scalable apps from scratch while the developer oversees the implementation.
Devin - An autonomous AI software engineer by Cognition Labs.
OpenDevin - An autonomous agent designed to navigate the complexities of software engineering. #opensource
Davika - An agentic AI software engineer. #opensource

Custom assistants

Poe - Poe gives access to a variety of bots.
GPT Builder - Assistant for creating GPT-based assistants.
GPTStore - Find useful GPTs. Share your own GPTs.

Image

Models

DALL·E 2 - DALL·E 2 by OpenAI is a new AI system that can create realistic images and art from a description in natural language.
Stable Diffusion - Stable Diffusion by Stability AI is a state of the art text-to-image model that generates images from text. #opensource
Midjourney - Midjourney is an independent research lab exploring new mediums of thought and expanding the imaginative powers of the human species.
Imagen - Imagen by Google is a text-to-image diffusion model with an unprecedented degree of photorealism and a deep level of language understanding.
Make-A-Scene - Make-A-Scene by Meta is a multimodal generative AI method puts creative control in the hands of people who use it by allowing them to describe and illustrate their vision through both text descriptions and freeform sketches.
DragGAN - Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold.

Services

Craiyon - Craiyon, formerly DALL-E mini, is an AI model that can draw images from any text prompt.
DreamStudio - DreamStudio is an easy-to-use interface for creating images using the Stable Diffusion image generation model.
Artbreeder - Artbreeder is new type of creative tool that empowers users creativity by making it easier to collaborate and explore.
GauGAN2 - GauGAN2 is a robust tool for creating photorealistic art using a combination of words and drawings since it integrates segmentation mapping, inpainting, and text-to-image production in a single model.
Magic Eraser - Remove unwanted things from images in seconds.
Imagine by Magic Studio - A tool by Magic Studio that let’s you express yourself by just describing what’s on your mind.
Alpaca - Stable Diffusion Photoshop plugin.
Patience.ai - Patience.ai is an app for creating images with Stable Diffusion, a cutting edge AI developed by Stability.AI.
GenShare - Generate art in seconds for free. Own and share what you create. A multimedia generative studio, democratizing design and creativity.
Playground - Playground is a free-to-use online AI image creator. Use it to create art, social media posts, presentations, posters, videos, logos and more.
Pixelz AI Art Generator - Pixelz AI Art Generator enables you to create incredible art from text. Stable Diffusion, CLIP Guided Diffusion & PXL·E realistic algorithms available.
modyfi - The image editor you’ve always wanted. AI-powered creative tools in your browser. Real-time collaboration.
Ponzu - Ponzu is your free AI logo generator. Build your brand with creatively designed logos in seconds, using only your imagination.
PhotoRoom - Create product and portrait pictures using only your phone. Remove background, change background and showcase products.
Avatar AI - Create your own AI-generated avatars.
ClipDrop - Create professional visuals without a photo studio, powered by stability.ai.
Lensa - An all-in-one image editing app that includes the generation of personalized avatars using Stable Diffusion.
RunDiffusion - Cloud-based workspace for creating AI-generated art.
Ideogram - A text-to-image platform to make creative expression more accessible.
Bing Image Creator - DALLE·3 based text-to-image generator with safety features.
KREA - Generate high quality visuals with an AI that knows about your styles, concepts, or products.
Nightcafe - NightCafe Creator is an AI Art Generator app with multiple methods of AI art generation.
Leonardo AI - Create production-quality visual assets for your projects with unprecedented quality, speed, and style.
Recraft - An AI tool that lets creators easily generate and iterate original images, vector art, illustrations, icons, and 3D graphics.
Reve Image - A model trained from the ground up to excel at prompt adherence, aesthetics, and typography.

Graphic design

Brandmark - AI-based logo design tool.
Gamma - Create beautiful presentations and webpages with none of the formatting and design work.
Microsoft Designer - Stunning designs in a flash.

Image libraries

Lexica - Stable Diffusion search engine.
OpenArt - Search 10M+ of prompts, and generate AI art via Stable Diffusion, DALL·E 2.
PromptHero - Search prompts for models like Stable Diffusion, ChatGPT, Midjourney, etc.
PromptBase - Search prompts from top prompt engineers. Sell your own prompts.

Model libraries

Civitai - Community driven AI model sharing tool.
Stable Diffusion Models - A comprehensive list of Stable Diffusion checkpoints on rentry.org.

Stable Diffusion resources

Stable Horde - A crowdsourced distributed cluster of Stable Diffusion workers.
DiffusionDB - A list of all public apps, developer tools, guides and plugins for Stable Diffusion. Airtable version.
PublicPrompts - A collection of free prompts for Stable Diffusion.
Stableboost - Stableboost is a Stable Diffusion WebUI that lets you quickly generate a lot of images so you can find the perfect ones.
Hugging Face Diffusion Models Course - Python materials for the online course on diffusion models by @huggingface.

Video

Runway - Magical AI tools, realtime collaboration, precision editing, and more. Your next-generation content creation suite.
Synthesia - Create videos from plain text in minutes.
Rephrase AI - Rephrase’s technology enables hyper-personalized video creation at scale that drive engagement and business efficiencies.
Hour One - Turn text into video, featuring virtual presenters, automatically.
Colossyan - Learning & Development focused video creator. Use AI avatars to create educational videos in multiple languages.
Fliki - Create text to video and text to speech content with ai powered voices in minutes.
Pictory - Pictory’s powerful AI enables you to create and edit professional quality videos using text.
Pika - An idea-to-video platform that brings your creativity to motion.
Sora - An AI model that can create realistic and imaginative scenes from text instructions.
Luma Dream Machine - An AI model that makes high quality, realistic videos fast from text and images.
Infinity AI - Infinity is a video foundation model that allows you to craft your characters and then bring them to life.
KLING AI - Tools for creating imaginative images and videos.
Hailuo AI - AI-powered text-to-video generator.

Avatars

D-ID - Create and interact with talking avatars at the touch of a button.
HeyGen - Turn scripts into talking videos with customizable AI avatars in minutes.
RenderNet - RenderNet AI is a tool for generating images and videos, providing control over character design, composition, and style.

Animation

Wonder Dynamics - Effortlessly animate, light, and compose CG characters into live scenes.

Audio

Text-to-speech

Eleven Labs - AI voice generator.
Resemble AI - AI voice generator and voice cloning for text to speech.
WellSaid - Convert text to voice in real time.
Play.ht - AI Voice Generator. Generate realistic Text to Speech voice over online with AI. Convert text to audio.
podcast.ai - A podcast that is entirely generated by artificial intelligence, powered by Play.ht text-to-voice AI.
VALL-E X - A cross-lingual neural codec language model for cross-lingual speech synthesis.
TorToiSe - A multi-voice text-to-speech system trained with an emphasis on quality. #opensource
Bark - A transformer-based text-to-audio model. #opensource

Speech-to-text

Whisper - Robust speech recognition via large-scale weak supervision. #opensource
Wispr Flow - Flow makes writing quick with seamless voice dictation for any application on your computer.
Vibe Transcribe - All-in-one solution for effortless audio and video transcription. #opensource
whisper.cpp - Port of OpenAI’s Whisper model in C/C++. #opensource

Music

Harmonai - We are a community-driven organization releasing open-source generative audio tools to make music production more accessible and fun for everyone.
Mubert - A royalty-free music ecosystem for content creators, brands and developers.
MusicLM - A model by Google Research for generating high-fidelity music from text descriptions.
AudioCraft - A single-stop code base for generative audio needs, by Meta. Includes MusicGen for music and AudioGen for sounds. #opensource
Stable Audio - Stable Audio is Stability AI’s first product for music and sound effect generation.
AIVA - AI-based music generation assistant. Choose from 250+ styles.
Suno AI - Anyone can make great music. No instrument needed, just imagination. From your mind to music.
Udio - Discover, create, and share music with the world.

Other

Diagram - Magical new ways to design products.
PromptBase - A marketplace for buying and selling quality prompts for DALL·E, GPT-3, Midjourney, Stable Diffusion.
This Image Does Not Exist - Test your ability to tell if an image is human or computer generated.
Have I Been Trained? - Check if your image has been used to train popular AI art models.
AI Dungeon - A text-based adventure-story game you direct (and star in) while the AI brings it to life.
Clickable - Generate ads in seconds with AI. Beautiful, brand-consistent, and highly converting ads for all marketing channels.
Scale Spellbook - Build, compare, and deploy large language model apps with Scale Spellbook.
Scenario - AI-generated gaming assets.
Teleprompter - An on-device AI for your meetings that listens to you and makes charismatic quote suggestions.
FinChat - Using AI, FinChat generates answers to questions about public companies and investors.
Morpher AI - Morpher AI delivers real-time insights and analysis for any market.
Whimsical AI - GPT-powered mind mapping, flowcharts, and visual tools for rapid idea development and process organization.

Learning resources

Learn Prompting - A free, open source course on communicating with artificial intelligence.
Prompt Engineering Guide - Guide and resources for prompt engineering.
ChatGPT prompt engineering for developers - A short course by Isa Fulford (OpenAI) and Andrew Ng (DeepLearning.AI).
OpenAI Cookbook - Examples and guides for using the OpenAI API.
OpenAI Prompt Engineering Guide - Strategies and tactics for getting better results from large language models.
PromptPerfect - Tool for prompt engineering.
Anthropic courses - Anthropic’s educational courses.
Build a Large Language Model (From Scratch) - A guide to building your own working LLM, by Sebastian Raschka.

More lists

Tools and Resources for AI Art - A large list of Google Colab notebooks for generative AI, by @pharmapsychotic.
The Generative AI Application Landscape - An infographic that maps the generative AI ecosystem, by Sonya Huang of Sequioa Capital.
Startups - @builtwithgenai - An Airtable list by @builtwithgenai.
The Generative AI Index - An Airtable list by Scale Venture Partners.
Generative AI for Games - A market map of companies working on Generative AI for games, by a16z.
Generative Deep Art - A curated list of generative deep learning tools, works, models, etc. for artistic uses, by @filipecalegario.
GPT-3 Demo - Showcase with GPT-3 examples, demos, apps, showcase, and NLP use-cases.
GPT-4 Demo - GPT-4 apps and use-cases.
The Generative AI Landscape - A Collection of Awesome Generative AI Applications.
Molecular design - List of molecular design using Generative AI and Deep Learning.
Open LLMs - A list of open LLMs available for commercial use.

Lists on ChatGPT

Awesome ChatGPT - A curated list of awesome tools, demos, docs for ChatGPT and GPT-3, by @jordn.
Awesome ChatGPT Prompts - A collection of prompt examples to be used with the ChatGPT model.
FlowGPT - Amplify your workflow with the best prompts.
ChatGPT Prompts for Data Science - A repository of useful data science prompts for ChatGPT.
Awesome ChatGPT - Another awesome list for ChatGPT.

go-genai

Mon, 02 Jun 2025 15:32:35 +0800

googleapis/go-genai

✨ NEW ✨

Google Gemini Multimodal Live support

Introducing support for the Gemini Multimodal Live feature. Here’s an example Multimodal Live server showing realtime conversation and video streaming: code

Google Gen AI Go SDK

The Google Gen AI Go SDK provides an interface for developers to integrate Google’s generative models into their Go applications. It supports the Gemini Developer API and Vertex AI APIs.

The Google Gen AI Go SDK enables developers to use Google’s state-of-the-art generative AI models (like Gemini) to build AI-powered features and applications. This SDK supports use cases like:

Generate text from text-only input
Generate text from text-and-images input (multimodal)
…

For example, with just a few lines of code, you can access Gemini’s multimodal capabilities to generate text from text-and-image input.

parts := []*genai.Part{
  {Text: "What's this image about?"},
  {InlineData: &genai.Blob{Data: imageBytes, MIMEType: "image/jpeg"}},
}
result, err := client.Models.GenerateContent(ctx, "gemini-2.0-flash", []*genai.Content{{Parts: parts}}, nil)

Installation and usage

Add the SDK to your module with go get google.golang.org/genai.

Create Clients

Imports

`1`	`import "google.golang.org/genai"`

Gemini API Client:

client, err := genai.NewClient(ctx, &genai.ClientConfig{
	APIKey:   apiKey,
	Backend:  genai.BackendGeminiAPI,
})

Vertex AI Client:

client, err := genai.NewClient(ctx, &genai.ClientConfig{
	Project:  project,
	Location: location,
	Backend:  genai.BackendVertexAI,
})

License

The contents of this repository are licensed under the Apache License, version 2.0.

torchtitan

Wed, 14 May 2025 15:30:19 +0800

pytorch/torchtitan

torchtitan

A PyTorch native platform for training generative AI models

[

torchtitan is currently in a pre-release state and under extensive development. We showcase training Llama 3.1 LLMs at scale, and are working on other types of generative AI models, including LLMs with MoE architectures, multimodal LLMs, and diffusion models, in the experiments folder. To use the latest features of torchtitan, we recommend using the most recent PyTorch nightly.

Latest News

[2025/04] Our paper has been accepted by ICLR 2025. The poster will be presented on Friday April 25th.
[2025/04] Llama 4 initial support is available as an experiment.
[2025/04] Training the diffusion model FLUX with FSDP/HSDP is available as an experiment.
[2025/04] The frontend implementation of SimpleFSDP, a compiler-based FSDP framework, is available as an experiment.
[2024/12] GPU MODE lecture on torchtitan.
[2024/11] Presentation at an AI/ML Infra Meetup.
[2024/07] Presentation at PyTorch Conference 2024.
[2024/04] Intro video - learn more about torchtitan in under 4 minutes.

Overview

torchtitan is a PyTorch native platform designed for rapid experimentation and large-scale training of generative AI models. As a minimal clean-room implementation of PyTorch native scaling techniques, torchtitan provides a flexible foundation for developers to build upon. With torchtitan extension points, one can easily create custom extensions tailored to specific needs.

Our mission is to accelerate innovation in the field of generative AI by empowering researchers and developers to explore new modeling architectures and infrastructure techniques.

The guiding principles when building torchtitan

Designed to be easy to understand, use and extend for different training purposes.
Minimal changes to the model code when applying multi-dimensional parallelism.
Bias towards a clean, minimal codebase while providing basic reusable / swappable components.

torchtitan has been showcasing PyTorch’s latest distributed training features, via pretraining Llama 3.1 LLMs of various sizes. To accelerate contributions to and innovations around torchtitan, we are hosting a new experiments folder. We look forward to your contributions!

Llama 3.1 pretraining

Key features available

Multi-dimensional composable parallelisms
- FSDP2 with per-parameter sharding
- Tensor Parallel (including async TP)
- Pipeline Parallel
- Context Parallel
Meta device initialization
Selective (layer or operator) and full activation checkpointing
Distributed checkpointing (including async checkpointing)
- Interoperable checkpoints which can be loaded directly into torchtune for fine-tuning
torch.compile support
Float8 support (how-to)
DDP and HSDP
TorchFT integration
Checkpointable data-loading, with the C4 dataset pre-configured (144M entries) and support for custom datasets
Flexible learning rate scheduler (warmup-stable-decay)
Loss, GPU memory, throughput (tokens/sec), TFLOPs, and MFU displayed and logged via Tensorboard or Weights & Biases
Debugging tools including CPU/GPU profiling, memory profiling, Flight Recorder, etc.
All options easily configured via toml files
Helper scripts to
- download tokenizers from Hugging Face
- convert original Llama 3 checkpoints into the expected DCP format
- estimate FSDP/HSDP memory usage without materializing the model
- run distributed inference with Tensor Parallel

We report performance on up to 512 GPUs, and verify loss converging correctness of various techniques.

Dive into the code

You may want to see how the model is defined or how parallelism techniques are applied. For a guided tour, see these files first:

torchtitan/train.py - the main training loop and high-level setup code
torchtitan/models/llama3/model.py - the Llama 3.1 model definition
torchtitan/models/llama3/parallelize_llama.py - helpers for applying Data Parallel, Tensor Parallel, activation checkpointing, and torch.compile to the model
torchtitan/models/llama3/pipeline_llama.py - helpers for applying Pipeline Parallel to the model
torchtitan/components/checkpoint.py - utils for saving/loading distributed checkpoints
torchtitan/components/float8.py - utils for applying Float8 techniques

Installation

git clone https://github.com/pytorch/torchtitan
cd torchtitan
pip install -r requirements.txt
pip3 install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu126 --force-reinstall
[For AMD GPU] pip3 install --pre torch --index-url https://download.pytorch.org/whl/nightly/rocm6.3 --force-reinstall

Downloading a tokenizer

torchtitan currently supports training Llama 3.1 (8B, 70B, 405B) out of the box. To get started training these models, we need to download a tokenizer.model. Follow the instructions on the official meta-llama repository to ensure you have access to the Llama model weights.

Once you have confirmed access, you can run the following command to download the Llama 3.1 tokenizer to your local machine.

# Get your HF token from https://huggingface.co/settings/tokens

# Llama 3.1 tokenizer.model
python scripts/download_tokenizer.py --repo_id meta-llama/Meta-Llama-3.1-8B --tokenizer_path "original" --hf_token=...

Start a training run

Llama 3 8B model locally on 8 GPUs

`1`	`CONFIG_FILE="./torchtitan/models/llama3/train_configs/llama3_8b.toml" ./run_train.sh`

Multi-Node Training

For training on ParallelCluster/Slurm type configurations, you can use the multinode_trainer.slurm file to submit your sbatch job.

To get started adjust the number of nodes and GPUs

1
2

#SBATCH --ntasks=2
#SBATCH --nodes=2

Then start a run where nnodes is your total node count, matching the sbatch node count above.

`1`	`srun torchrun --nnodes 2`

If your gpu count per node is not 8, adjust --nproc_per_node in the torchrun command and #SBATCH --gpus-per-task in the SBATCH command section.

Citation

We provide a detailed look into the parallelisms and optimizations available in torchtitan, along with summary advice on when to use various techniques.

TorchTitan: One-stop PyTorch native solution for production ready LLM pre-training

@inproceedings{
   liang2025torchtitan,
   title={TorchTitan: One-stop PyTorch native solution for production ready {LLM} pretraining},
   author={Wanchao Liang and Tianyu Liu and Less Wright and Will Constable and Andrew Gu and Chien-Chin Huang and Iris Zhang and Wei Feng and Howard Huang and Junjie Wang and Sanket Purandare and Gokul Nadathur and Stratos Idreos},
   booktitle={The Thirteenth International Conference on Learning Representations},
   year={2025},
   url={https://openreview.net/forum?id=SFN6Wm7YBI}
}

License

Source code is made available under a BSD 3 license, however you may have other legal obligations that govern your use of other content linked in this repository, such as the license or terms of service for third-party data and models.

GenAI-Showcase

Sat, 12 Apr 2025 15:26:11 +0800

mongodb-developer/GenAI-Showcase

MongoDB’s GenAI Showcase

Welcome to MongoDB’s Generative AI Showcase Repository!

Whether you are just starting out on your Generative AI journey, or looking to build advanced GenAI applications, we’ve got you covered. This repository has an exhaustive list of examples and sample applications that cover Retrieval-Augmented Generation (RAG), AI Agents, and industry-specific use cases.

Discover how MongoDB integrates into RAG pipelines and AI Agents, serving as a vector database, operational database, and memory provider.

This repo mainly contains:

Folder	Description
`notebooks`	Jupyter notebooks examples for RAG, agentic applications, evaluations etc.
`apps`	Javascipt and Python apps and demos
`partners`	Contributions from our AI partners

Getting Started

You will need to connect to a MongoDB cluster to run any of the apps or examples in this repo. Follow these steps to get set up:

Register for a free MongoDB Atlas account
Create a new database cluster
Obtain the connection string for your database cluster

Contributing

We welcome contributions! Please read our Contribution Guidelines for more information on how to participate.

License

This project is licensed under the MIT License.

Getting Support

As you work through these examples, if you encounter any problems, please open a new issue.

Additional Resources

llm-cookbook

Fri, 11 Apr 2025 15:27:55 +0800

datawhalechina/llm-cookbook

面向开发者的大模型手册 - LLM Cookbook

项目简介

本项目是一个面向开发者的大模型手册，针对国内开发者的实际需求，主打 LLM 全方位入门实践。本项目基于吴恩达老师大模型系列课程内容，对原课程内容进行筛选、翻译、复现和调优，覆盖从 Prompt Engineering 到 RAG 开发、模型微调的全部流程，用最适合国内学习者的方式，指导国内开发者如何学习、入门 LLM 相关项目。

针对不同内容的特点，我们对共计 11 门吴恩达老师的大模型课程进行了翻译复现，并结合国内学习者的实际情况，对不同课程进行了分级和排序，初学者可以先系统学习我们的必修类课程，掌握入门 LLM 所有方向都需要掌握的基础技能和概念，再选择性地学习我们的选修类课程，在自己感兴趣的方向上不断探索和学习。

如果有你非常喜欢但我们还没有进行复现的吴恩达老师大模型课程，我们欢迎每一位开发者参考我们已有课程的格式和写法来对课程进行复现并提交 PR，在 PR 审核通过后，我们会根据课程内容将课程进行分级合并。欢迎每一位开发者的贡献！

在线阅读地址：面向开发者的 LLM 入门课程-在线阅读

PDF下载地址：面向开发者的 LLM 入门教程-PDF

英文原版地址：吴恩达关于大模型的系列课程

项目意义

LLM 正在逐步改变人们的生活，而对于开发者，如何基于 LLM 提供的 API 快速、便捷地开发一些具备更强能力、集成LLM 的应用，来便捷地实现一些更新颖、更实用的能力，是一个急需学习的重要能力。

由吴恩达老师与 OpenAI 合作推出的大模型系列教程，从大模型时代开发者的基础技能出发，深入浅出地介绍了如何基于大模型 API、LangChain 架构快速开发结合大模型强大能力的应用。其中，《Prompt Engineering for Developers》教程面向入门 LLM 的开发者，深入浅出地介绍了对于开发者，如何构造 Prompt 并基于 OpenAI 提供的 API 实现包括总结、推断、转换等多种常用功能，是入门 LLM 开发的经典教程；《Building Systems with the ChatGPT API》教程面向想要基于 LLM 开发应用程序的开发者，简洁有效而又系统全面地介绍了如何基于 ChatGPT API 打造完整的对话系统；《LangChain for LLM Application Development》教程结合经典大模型开源框架 LangChain，介绍了如何基于 LangChain 框架开发具备实用功能、能力全面的应用程序，《LangChain Chat With Your Data》教程则在此基础上进一步介绍了如何使用 LangChain 架构结合个人私有数据开发个性化大模型应用；《Building Generative AI Applications with Gradio》、《Evaluating and Debugging Generative AI》教程分别介绍了两个实用工具 Gradio 与 W&B，指导开发者如何结合这两个工具来打造、评估生成式 AI 应用。

上述教程非常适用于开发者学习以开启基于 LLM 实际搭建应用程序之路。因此，我们将该系列课程翻译为中文，并复现其范例代码，也为其中一个视频增加了中文字幕，支持国内中文学习者直接使用，以帮助中文学习者更好地学习 LLM 开发；我们也同时实现了效果大致相当的中文 Prompt，支持学习者感受中文语境下 LLM 的学习使用，对比掌握多语言语境下的 Prompt 设计与 LLM 开发。未来，我们也将加入更多 Prompt 高级技巧，以丰富本课程内容，帮助开发者掌握更多、更巧妙的 Prompt 技能。

项目受众

所有具备基础 Python 能力，想要入门 LLM 的开发者。

项目亮点

《ChatGPT Prompt Engineering for Developers》、《Building Systems with the ChatGPT API》等教程作为由吴恩达老师与 OpenAI 联合推出的官方教程，在可预见的未来会成为 LLM 的重要入门教程，但是目前还只支持英文版且国内访问受限，打造中文版且国内流畅访问的教程具有重要意义；同时，GPT 对中文、英文具有不同的理解能力，本教程在多次对比、实验之后确定了效果大致相当的中文 Prompt，支持学习者研究如何提升 ChatGPT 在中文语境下的理解与生成能力。

学习指南

本教程适用于所有具备基础 Python 能力，想要入门 LLM 的开发者。

如果你想要开始学习本教程，你需要提前具备：

至少一个 LLM API（最好是 OpenAI，如果是其他 API，你可能需要参考其他教程对 API 调用代码进行修改）
能够使用 Python Jupyter Notebook

本教程共包括 11 门课程，分为必修类、选修类两个类别。必修类课程是我们认为最适合初学者学习以入门 LLM 的课程，包括了入门 LLM 所有方向都需要掌握的基础技能和概念，我们也针对必修类课程制作了适合阅读的在线阅读和 PDF 版本，在学习必修类课程时，我们建议学习者按照我们列出的顺序进行学习；选修类课程是在必修类课程上的拓展延伸，包括了 RAG 开发、模型微调、模型评估等多个方面，适合学习者在掌握了必修类课程之后选择自己感兴趣的方向和课程进行学习。

必修类课程包括：

面向开发者的 Prompt Engineering。基于吴恩达老师《ChatGPT Prompt Engineering for Developers》课程打造，面向入门 LLM 的开发者，深入浅出地介绍了对于开发者，如何构造 Prompt 并基于 OpenAI 提供的 API 实现包括总结、推断、转换等多种常用功能，是入门 LLM 开发的第一步。
搭建基于 ChatGPT 的问答系统。基于吴恩达老师《Building Systems with the ChatGPT API》课程打造，指导开发者如何基于 ChatGPT 提供的 API 开发一个完整的、全面的智能问答系统。通过代码实践，实现了基于 ChatGPT 开发问答系统的全流程，介绍了基于大模型开发的新范式，是大模型开发的实践基础。
使用 LangChain 开发应用程序。基于吴恩达老师《LangChain for LLM Application Development》课程打造，对 LangChain 展开深入介绍，帮助学习者了解如何使用 LangChain，并基于 LangChain 开发完整的、具备强大能力的应用程序。
使用 LangChain 访问个人数据。基于吴恩达老师《LangChain Chat with Your Data》课程打造，深入拓展 LangChain 提供的个人数据访问能力，指导开发者如何使用 LangChain 开发能够访问用户个人数据、提供个性化服务的大模型应用。

选修类课程包括：

使用 Gradio 搭建生成式 AI 应用。基于吴恩达老师《Building Generative AI Applications with Gradio》课程打造，指导开发者如何使用 Gradio 通过 Python 接口程序快速、高效地为生成式 AI 构建用户界面。
评估改进生成式 AI。基于吴恩达老师《Evaluating and Debugging Generative AI》课程打造，结合 wandb，提供一套系统化的方法和工具，帮助开发者有效地跟踪和调试生成式 AI 模型。
微调大语言模型。基于吴恩达老师《Finetuning Large Language Model》课程打造，结合 lamini 框架，讲述如何便捷高效地在本地基于个人数据微调开源大语言模型。
大模型与语义检索。基于吴恩达老师《Large Language Models with Semantic Search》课程打造，针对检索增强生成，讲述了多种高级检索技巧以实现更准确、高效的检索增强 LLM 生成效果。
基于 Chroma 的高级检索。基于吴恩达老师《Advanced Retrieval for AI with Chroma》课程打造，旨在介绍基于 Chroma 的高级检索技术，提升检索结果的准确性。
搭建和评估高级 RAG 应用。基于吴恩达老师《Building and Evaluating Advanced RAG Applications》课程打造，介绍构建和实现高质量RAG系统所需的关键技术和评估框架。
LangChain 的 Functions、Tools 和 Agents。基于吴恩达老师《Functions, Tools and Agents with LangChain》课程打造，介绍如何基于 LangChain 的新语法构建 Agent。
Prompt 高级技巧。包括 CoT、自我一致性等多种 Prompt 高级技巧的基础理论与代码实现。

其他资料包括：

双语字幕视频地址：吴恩达 x OpenAI的Prompt Engineering课程专业翻译版

中英双语字幕下载：《ChatGPT提示工程》非官方版中英双语字幕

视频讲解：面向开发者的 Prompt Engineering 讲解（数字游民大会）

目录结构说明：

content：基于原课程复现的双语版代码，可运行的 Notebook，更新频率最高，更新速度最快。

docs：必修类课程文字教程版在线阅读源码，适合阅读的 md。

figures：图片文件。

致谢

核心贡献者

邹雨衡-项目负责人（Datawhale成员-对外经济贸易大学研究生）
左春生-项目负责人（内容创作者-Datawhale成员）
长琴-项目发起人（内容创作者-Datawhale成员-AI算法工程师）
玉琳-项目发起人（内容创作者-Datawhale成员）
徐虎-教程编撰者（内容创作者-Datawhale成员）
刘伟鸿-教程编撰者（内容创作者-江南大学非全研究生）
Joye-教程编撰者（内容创作者-数据科学家）
高立业（内容创作者-DataWhale成员-算法工程师）
邓宇文（内容创作者-Datawhale成员）
魂兮（内容创作者-前端工程师）
宋志学（内容创作者-Datawhale成员）
韩颐堃（内容创作者-Datawhale成员）
陈逸涵 (内容创作者-Datawhale意向成员-AI爱好者)
仲泰（内容创作者-Datawhale成员）
万礼行（内容创作者-视频翻译者）
王熠明（内容创作者-Datawhale成员）
曾浩龙（内容创作者-Datawhale 意向成员-JLU AI 研究生）
小饭同学（内容创作者）
孙韩玉（内容创作者-算法量化部署工程师）
张银晗（内容创作者-Datawhale成员）
张晋（内容创作者-Datawhale成员）
李娇娇（内容创作者-Datawhale成员）
邓恺俊（内容创作者-Datawhale成员）
范致远（内容创作者-Datawhale成员）
周景林（内容创作者-Datawhale成员）
诸世纪（内容创作者-算法工程师）
Zhang Yixin（内容创作者-IT爱好者）
Sarai（内容创作者-AI应用爱好者）

其他

特别感谢 @Sm1les、@LSGOMYP 对本项目的帮助与支持；
感谢 GithubDaily 提供的双语字幕；
如果有任何想法可以联系我们 DataWhale 也欢迎大家多多提出 issue；
特别感谢以下为教程做出贡献的同学！

Made with contrib.rocks.

Star History

关注我们

扫描下方二维码关注公众号：Datawhale

Datawhale 是一个专注于数据科学与 AI 领域的开源组织，汇集了众多领域院校和知名企业的优秀学习者，聚合了一群有开源精神和探索精神的团队成员。微信搜索公众号Datawhale可以加入我们。

LICENSE

本作品采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可。

llama-models

Tue, 08 Apr 2025 15:27:36 +0800

meta-llama/llama-models

🤗 Models on Hugging Face | Blog | Website | Get Started | Llama Cookbook

Llama Models

Llama is an accessible, open large language model (LLM) designed for developers, researchers, and businesses to build, experiment, and responsibly scale their generative AI ideas. Part of a foundational system, it serves as a bedrock for innovation in the global community. A few key aspects:

Open access: Easy accessibility to cutting-edge large language models, fostering collaboration and advancements among developers, researchers, and organizations
Broad ecosystem: Llama models have been downloaded hundreds of millions of times, there are thousands of community projects built on Llama and platform support is broad from cloud providers to startups - the world is building with Llama!
Trust & safety: Llama models are part of a comprehensive approach to trust and safety, releasing models and tools that are designed to enable community collaboration and encourage the standardization of the development and usage of trust and safety tools for generative AI

Our mission is to empower individuals and industry through this opportunity while fostering an environment of discovery and ethical AI advancements. The model weights are licensed for researchers and commercial entities, upholding the principles of openness.

Llama Models

Model	Launch date	Model sizes	Context Length	Tokenizer	Acceptable use policy	License	Model Card
Llama 2	7/18/2023	7B, 13B, 70B	4K	Sentencepiece	Use Policy	License	Model Card
Llama 3	4/18/2024	8B, 70B	8K	TikToken-based	Use Policy	License	Model Card
Llama 3.1	7/23/2024	8B, 70B, 405B	128K	TikToken-based	Use Policy	License	Model Card
Llama 3.2	9/25/2024	1B, 3B	128K	TikToken-based	Use Policy	License	Model Card
Llama 3.2-Vision	9/25/2024	11B, 90B	128K	TikToken-based	Use Policy	License	Model Card
Llama 3.3	12/04/2024	11B, 90B	128K	TikToken-based	Use Policy	License	Model Card
Llama 4	4/5/2025	Scout-17B-16E, Maverick-17B-128E	10M, 1M	TikToken-based	Use Policy	License	Model Card

Download

To download the model weights and tokenizer:

Visit the Meta Llama website.
Read and accept the license.
Once your request is approved you will receive a signed URL via email.
Install the Llama CLI: pip install llama-stack. (<– Start Here if you have received an email already.)
Run llama model list to show the latest available models and determine the model ID you wish to download. NOTE: If you want older versions of models, run llama model list --show-all to show all the available Llama models.
Run: llama download --source meta --model-id CHOSEN_MODEL_ID
Pass the URL provided when prompted to start the download.

Remember that the links expire after 24 hours and a certain amount of downloads. You can always re-request a link if you start seeing errors such as 403: Forbidden.

Running the models

In order to run the models, you will need to install dependencies after checking out the repository.

1
2

# Run this within a suitable Python environment (uv, conda, or virtualenv)
pip install .[torch]

Example scripts are available in models/{ llama3, llama4 }/scripts/ sub-directory. Note that the Llama4 series of models require at least 4 GPUs to run inference at full (bf16) precision.

#!/bin/bash

NGPUS=4
CHECKPOINT_DIR=~/.llama/checkpoints/Llama-4-Scout-17B-16E-Instruct
PYTHONPATH=$(git rev-parse --show-toplevel) \
  torchrun --nproc_per_node=$NGPUS \
  -m models.llama4.scripts.chat_completion $CHECKPOINT_DIR \
  --world_size $NGPUS

The above script should be used with an Instruct (Chat) model. For a Base model, update the CHECKPOINT_DIR path and use the script models.llama4.scripts.completion.

Running inference with FP8 and Int4 Quantization

You can reduce the memory footprint of the models at the cost of minimal loss in accuracy by running inference with FP8 or Int4 quantization. Use the --quantization-mode flag to specify the quantization mode. There are two modes:

fp8_mixed: Mixed precision inference with FP8 for some weights and bfloat16 for activations.
int4_mixed: Mixed precision inference with Int4 for some weights and bfloat16 for activations.

Using FP8, running Llama-4-Scout-17B-16E-Instruct requires 2 GPUs with 80GB of memory. Using Int4, you need a single GPU with 80GB of memory.

MODE=fp8_mixed  # or int4_mixed
if [ $MODE == "fp8_mixed" ]; then
  NGPUS=2
else
  NGPUS=1
fi
CHECKPOINT_DIR=~/.llama/checkpoints/Llama-4-Scout-17B-16E-Instruct
PYTHONPATH=$(git rev-parse --show-toplevel) \
  torchrun --nproc_per_node=$NGPUS \
  -m models.llama4.scripts.chat_completion $CHECKPOINT_DIR \
  --world_size $NGPUS \
  --quantization-mode $MODE

For more flexibility in running inference (including using other providers), please see the Llama Stack toolset.

Access to Hugging Face

We also provide downloads on Hugging Face, in both transformers and native llama4 formats. To download the weights from Hugging Face, please follow these steps:

Visit one of the repos, for example meta-llama/Llama-4-Scout-17B-16E.
Read and accept the license. Once your request is approved, you’ll be granted access to all Llama 3.1 models as well as previous versions. Note that requests used to take up to one hour to get processed.
To download the original native weights to use with this repo, click on the “Files and versions” tab and download the contents of the original folder. You can also download them from the command line if you pip install huggingface-hub:

`1`	`huggingface-cli download meta-llama/Llama-4-Scout-17B-16E-Instruct-Original --local-dir meta-llama/Llama-4-Scout-17B-16E-Instruct-Original`

To use with transformers, the following snippet will download and cache the weights:

# inference.py
from transformers import AutoTokenizer, Llama4ForConditionalGeneration
import torch

model_id = "meta-llama/Llama-4-Scout-17B-16E-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
)

model = Llama4ForConditionalGeneration.from_pretrained(
    model_id, device_map="auto", torch_dtype=torch.bfloat16
)

outputs = model.generate(**inputs.to(model.device), max_new_tokens=100)
outputs = tokenizer.batch_decode(outputs[:, inputs["input_ids"].shape[-1] :])
print(outputs[0])

`1`	`torchrun --nnodes=1 --nproc_per_node=8 inference.py`

Installations

You can install this repository as a package by just doing pip install llama-models

Responsible Use

Llama models are a new technology that carries potential risks with use. Testing conducted to date has not — and could not — cover all scenarios. To help developers address these risks, we have created the Responsible Use Guide.

Issues

Please report any software “bug” or other problems with the models through one of the following means:

Reporting issues with the model: https://github.com/meta-llama/llama-models/issues
Reporting risky content generated by the model: developers.facebook.com/llama_output_feedback
Reporting bugs and security concerns: facebook.com/whitehat/info

Questions

For common questions, the FAQ can be found here, which will be updated over time as new questions arise.

Generative AI on Producthunt daily

gallery

Google AI Edge Gallery ✨

App Preview

✨ Core Features

🏁 Get Started in Minutes!

🛠️ Technology Highlights

⌨️ Development

🤝 Feedback

📄 License

🔗 Useful Links

generative-ai-for-beginners

21 Lessons teaching everything you need to know to start building Generative AI applications

🌐 Multi-Language Support

Supported via GitHub Action (Automated & Always Up-to-Date)

Generative AI for Beginners (Version 3) - A Course

🌱 Getting Started

What You Need

To run the code of this course, you can use either:

🧠 Ready to Deploy?

🗣️ Meet Other Learners, Get Support

🚀 Building a Startup?

🙏 Want to help?

📂 Each lesson includes:

🗃️ Lessons

🌟 Special thanks

🎒 Other Courses

awesome-generative-ai

Awesome Generative AI

Contents

Recommended reading

Milestones

Text

Models

Chatbots

Custom interfaces

Search engines

Local search engines

Writing assistants

ChatGPT extensions

Productivity

Meeting assistants

Academia

Leaderboards

Other text generators

Coding

Coding Assistants

Developer tools

Playgrounds

Local LLM Deployment

Agents

Autonomous agents

Custom assistants

Image

Models

Services

Graphic design

Image libraries

Model libraries

Stable Diffusion resources

Video

Avatars

Animation

Audio

Text-to-speech

Speech-to-text

Music

Other

Learning resources

More lists

Lists on ChatGPT

go-genai

✨ NEW ✨

Google Gemini Multimodal Live support

Google Gen AI Go SDK

Installation and usage

Create Clients

Imports

Gemini API Client:

Vertex AI Client: