Cloud Computing on Producthunt daily

puter

Fri, 22 Aug 2025 15:28:22 +0800

HeyPuter/puter

The Internet OS! Free, Open-Source, and Self-Hostable.

« LIVE DEMO »

Puter.com · App Store · Developers · CLI · Discord · Reddit · X

Puter

Puter is an advanced, open-source internet operating system designed to be feature-rich, exceptionally fast, and highly extensible. Puter can be used as:

A privacy-first personal cloud to keep all your files, apps, and games in one secure place, accessible from anywhere at any time.
A platform for building and publishing websites, web apps, and games.
An alternative to Dropbox, Google Drive, OneDrive, etc. with a fresh interface and powerful features.
A remote desktop environment for servers and workstations.
A friendly, open-source project and community to learn about web development, cloud computing, distributed systems, and much more!

Getting Started

💻 Local Development

git clone https://github.com/HeyPuter/puter
cd puter
npm install
npm start

→ This should launch Puter at http://puter.localhost:4100 (or the next available port).

If this does not work, see First Run Issues for troubleshooting steps.

🐳 Docker

mkdir puter && cd puter && mkdir -p puter/config puter/data && sudo chown -R 1000:1000 puter && docker run --rm -p 4100:4100 -v `pwd`/puter/config:/etc/puter -v `pwd`/puter/data:/var/puter  ghcr.io/heyputer/puter

→ This should launch Puter at http://puter.localhost:4100 (or the next available port).

🐙 Docker Compose

Linux/macOS

mkdir -p puter/config puter/data
sudo chown -R 1000:1000 puter
wget https://raw.githubusercontent.com/HeyPuter/puter/main/docker-compose.yml
docker compose up

→ This should be available at http://puter.localhost:4100 (or the next available port).

Windows

mkdir -p puter
cd puter
New-Item -Path "puter\config" -ItemType Directory -Force
New-Item -Path "puter\data" -ItemType Directory -Force
Invoke-WebRequest -Uri "https://raw.githubusercontent.com/HeyPuter/puter/main/docker-compose.yml" -OutFile "docker-compose.yml"
docker compose up

→ This should launch Puter at http://puter.localhost:4100 (or the next available port).

🚀 Self-Hosting

For detailed guides on self-hosting Puter, including configuration options and best practices, see our Self-Hosting Documentation.

☁️ Puter.com

Puter is available as a hosted service at puter.com.

System Requirements

Operating Systems: Linux, macOS, Windows
RAM: 2GB minimum (4GB recommended)
Disk Space: 1GB free space
Node.js: Version 16+ (Version 23+ recommended)
npm: Latest stable version

Support

Connect with the maintainers and community through these channels:

Bug report or feature request? Please open an issue.
Discord: discord.com/invite/PQcx7Teh8u
X (Twitter): x.com/HeyPuter
Reddit: reddit.com/r/puter/
Mastodon: mastodon.social/@puter
Security issues? security@puter.com
Email maintainers at hi@puter.com

We are always happy to help you with any questions you may have. Don’t hesitate to ask!

License

This repository, including all its contents, sub-projects, modules, and components, is licensed under AGPL-3.0 unless explicitly stated otherwise. Third-party libraries included in this repository may be subject to their own licenses.

Translations

data-engineer-handbook

Thu, 21 Aug 2025 15:31:15 +0800

DataExpert-io/data-engineer-handbook

The Data Engineering Handbook

This repo has all the resources you need to become an amazing data engineer!

Getting started

If you are new to data engineering, start by following this 2024 breaking into data engineering roadmap

If you are here for the 4-week free beginner boot camp you can check out:

If you are here for the 6-week free intermediate boot camp you can check out

For more applied learning:

Check out the projects section for more hands-on examples!
Check out the interviews section for more advice on how to pass data engineering interviews!
Check out the books section for a list of high quality data engineering books
Check out the communities section for a list of high quality data engineering communities to join
Check out the newsletter section to learn via email

Resources

Great list of over 25 books

Great list of over 10 communities to join:

Top must-join communities for DE:

Top must-join communities for ML:

Companies:

Orchestration
- Mage
- Astronomer
- Prefect
- Dagster
- Airflow
- Kestra
- Shipyard
- Hamilton
Data Lake / Cloud
- Tabular
- Microsoft
- Databricks
- Onehouse
- Delta Lake
- Ilum
- DuckLake
- Apache Iceberg
- Apache Polaris
- Lakekeeper
Data Warehouse
Data Quality
- dbt
- Metaplane
- Gable
- Great Expectations
- Streamdal
- Coalesce
- Soda
- DQOps
- HEDDA.IO
- Dingo
Education Companies
Analytics / Visualization
- Preset
- Starburst
- Metabase
- Looker Studio
- Tableau
- Power BI
- Hex
- Apache Superset
- Evidence
- Redash
- Lightdash
Data Integration
- Cube
- Fivetran
- Airbyte
- dlt
- Sling
- Meltano
- Estuary
Semantic Layers
- Cube
- dbt Semantic Layer
Modern OLAP
- Apache Druid
- ClickHouse
- Apache Pinot
- Apache Kylin
- DuckDB
- QuestDB
- StarRocks
LLM application library
Real-Time Data
Data Lineage
- OpenLineage

Data Engineering blogs of companies:

Data Engineering Whitepapers:

Here’s the mostly comprehensive list of data engineering creators: (You have to have at least 5k followers somewhere to be added!)

YouTube

Name	YouTube Channel	Follower Count
ByteByteGo	ByteByteGo	1,000,000+
Zach Wilson	Data with Zach	150,000+
Shashank Mishra	E-learning Bridge	100,000+
Seattle Data Guy	Seattle Data Guy	100,000+
TrendyTech	TrendyTech	100,000+
Darshil Parmar	Darshil Parmar	100,000+
Andreas Kretz	Andreas Kretz	100,000+
The Ravit Show	The Ravit Show	100,000+
Guy in a Cube	Guy in a Cube	100,000+
Adam Marczak	Adam Marczak	100,000+
nullQueries	nullQueries	100,000+
TECHTFQ by Thoufiq	TECHTFQ by Thoufiq	100,000+
SQLBI	SQLBI	100,000+
Alex Freberg	Alex The Analyst	100,000+
Ankur Ranjan	Big Data Show	100,000+
Prashanth Kumar Pandey	ScholarNest	77,000+
ITVersity	ITVersity	67,000+
Soumil Shah	Soumil Shah	50,000
Ansh Lamba	Ansh Lamba	18,000+
Azure Lib	Azure Lib	10,000+
Advancing Analytics	Advancing Analytics	10,000+
Kahan Data Solutions	Kahan Data Solutions	10,000+
Ankit Bansal	Ankit Bansal	10,000+
Mr. K Talks Tech	Mr. K Talks Tech	10,000+
Samuel Focht	Python Basics	10,000+
Mehdi Ouazza	Mehdio DataTV	3,000+
Alex Merced	Alex Merced Data	N/A
John Kutay	John Kutay	N/A
Emil Kaminski	Databricks For Professionals	5,000+

Name	LinkedIn Profile	Follower Count
Zach Wilson	Zach Wilson	400,000+
Chip Huyen	Chip Huyen	250,000+
Shashank Mishra	Shashank Mishra	100,000+
Seattle Data Guy	Ben Rogojan	100,000+
TrendyTech	Sumit Mittal	100,000+
Darshil Parmar	Darshil Parmar	100,000+
Andreas Kretz	Andreas Kretz	100,000+
ByteByteGo (Alex Xu)	Alex Xu	100,000+
Azure Lib (Deepak Goyal)	Deepak Goyal	100,000+
Alex Freberg	Alex Freberg	100,000+
SQLBI (Marco Russo)	Marco Russo	50,000+
Ankit Bansal	Ankit Bansal	50,000+
Marc Lamberti	Marc Lamberti	50,000+
Ankur Ranjan	Ankur Ranjan	48,000+
ITVersity (Durga Gadiraju)	Durga Gadiraju	48,000+
Prashanth Kumar Pandey	Prashanth Kumar Pandey	37,000+
Alex Merced	Alex Merced	30,000+
Ijaz Ali	Ijaz Ali	24,000+
Mehdi Ouazza	Mehdi Ouazza	20,000+
Ananth Packkildurai	Ananth Packkildurai	18,000+
Ansh Lamba	Ansh Lamba	13,000+
Manojkumar Vadivel	Manojkumar Vadivel	12,000+
Advancing Analytics	Simon Whiteley	10,000+
Li Yin	Li Yin	10,000+
Jaco van Gelder	Jaco van Gelder	10,000+
Joseph Machado	Joseph Machado	10,000+
Eric Roby	Eric Roby	10,000+
Simon Späti	Simon Späti	10,000+
Constantin Lungu	Constantin Lungu	10,000+
Lakshmi Sontenam	Lakshmi Sontenam	9,500+
Dani Pálma	Daniel Pálma	9,000+
Soumil Shah	Soumil Shah	8,000+
Arnaud Milleker	Arnaud Milleker	7,000+
Dimitri Visnadi	Dimitri Visnadi	7,000+
Lenny	Lenny A	6,000+
Dipankar Mazumdar	Dipankar Mazumdar	5,000+
Daniel Ciocirlan	Daniel Ciocirlan	5,000+
Hugo Lu	Hugo Lu	5,000+
Tobias Macey	Tobias Macey	5,000+
Marcos Ortiz	Marcos Ortiz	5,000+
Julien Hurault	Julien Hurault	5,000+
John Kutay	John Kutay	5,000+
Hassaan Akbar	Hassaan Akbar	5,000+
Subhankar	Subhankar	5,000+
Nitin	Nitin	N/A
Hassaan	Hassaan	5000+
Javier de la Torre	Javier	5000+

X/Twitter

Name	X/Twitter Profile	Follower Count
ByteByteGo	alexxubyte	100,000+
Dan Kornas	@dankornas	66,000+
Zach Wilson	EcZachly	30,000+
Seattle Data Guy	SeattleDataGuy	10,000+
SQLBI	marcorus	10,000+
Joseph Machado	startdataeng	5,000+
Alex Merced	@amdatalakehouse	N/A
John Kutay	@JohnKutay	N/A
Mehdi Ouazza	mehd_io	N/A

Instagram

Name	Instagram Profile	Follower Count
Sundas Khalid	sundaskhalidd	300,000+
Zach Wilson	eczachly	150,000+
Andreas Kretz	learndataengineering	5,000+
Alex Merced	@alexmercedcoder	N/A

TikTok

Name	TikTok Profile	Follower Count
Zach Wilson	@eczachly	70,000+
Alex Freberg	@alex_the_analyst	10,000+
Mehdi Ouazza	@mehdio_datatv	N/A

Great Podcasts

Great list of 20+ newsletters

Top must follow newsletters for data engineering:

Glossaries:

Design Patterns

Courses / Academies

DataExpert.io course use code HANDBOOK10 for a discount!
LearnDataEngineering.com
Technical Freelancer Academy Use code zwtech for a discount!
IBM Data Engineering for Everyone
Qwiklabs
DataCamp
Udemy Courses from Shruti Mantri
Rock the JVM teaches Spark (in Scala), Flink and others
Data Engineering Zoomcamp by DataTalksClub
Efficient Data Processing in Spark
Scaler
DataTeams - Data Engingeer hiring platform
Udemy Courses from Daniel Blanco

Certifications Courses

skypilot

Sun, 27 Apr 2025 15:27:09 +0800

skypilot-org/skypilot

Run AI on Any Infra — Unified, Faster, Cheaper

:fire: News :fire:

[Mar 2025] Run and serve Google Gemma 3 using SkyPilot example
[Feb 2025] Prepare and serve Retrieval Augmented Generation (RAG) with DeepSeek-R1: blog post, example
[Feb 2025] Run and serve DeepSeek-R1 671B using SkyPilot and SGLang with high throughput: example
[Feb 2025] Prepare and serve large-scale image search with vector databases: blog post, example
[Jan 2025] Launch and serve distilled models from DeepSeek-R1 and Janus on Kubernetes or any cloud: R1 example and Janus example
[Oct 2024] :tada: SkyPilot crossed 1M+ downloads :tada:: Thank you to our community! Twitter/X
[Sep 2024] Point, launch and serve Llama 3.2 on Kubernetes or any cloud: example
[Sep 2024] Run and deploy Pixtral, the first open-source multimodal model from Mistral AI.
[Jun 2024] Reproduce GPT with llm.c on any cloud: guide
[Apr 2024] Serve Qwen-110B on your infra: example
[Apr 2024] Host Ollama on the cloud to deploy LLMs on CPUs and GPUs: example

LLM Finetuning Cookbooks: Finetuning Llama 2 / Llama 3.1 in your own cloud environment, privately: Llama 2 example and blog; Llama 3.1 example and blog

SkyPilot is an open-source framework for running AI and batch workloads on any infra.

SkyPilot is easy to use for AI users:

Quickly spin up compute on your own infra
Environment and job as code — simple and portable
Easy job management: queue, run, and auto-recover many jobs

SkyPilot unifies multiple clusters, clouds, and hardware:

One interface to use reserved GPUs, Kubernetes clusters, or 16+ clouds
Flexible provisioning of GPUs, TPUs, CPUs, with auto-retry
Team deployment and resource sharing

SkyPilot cuts your cloud costs & maximizes GPU availability:

Autostop: automatic cleanup of idle resources
Spot instance support: 3-6x cost savings, with preemption auto-recovery
Intelligent scheduling: automatically run on the cheapest & most available infra

SkyPilot supports your existing GPU, TPU, and CPU workloads, with no code changes.

Install with pip:

1
2

# Choose your clouds:
pip install -U "skypilot[kubernetes,aws,gcp,azure,oci,lambda,runpod,fluidstack,paperspace,cudo,ibm,scp,nebius]"

To get the latest features and fixes, use the nightly build or install from source:

1
2

# Choose your clouds:
pip install "skypilot-nightly[kubernetes,aws,gcp,azure,oci,lambda,runpod,fluidstack,paperspace,cudo,ibm,scp,nebius]"

Current supported infra: Kubernetes, AWS, GCP, Azure, OCI, Lambda Cloud, Fluidstack, RunPod, Cudo, Digital Ocean, Paperspace, Cloudflare, Samsung, IBM, Vast.ai, VMware vSphere, Nebius.

Getting started

You can find our documentation here.

SkyPilot in 1 minute

A SkyPilot task specifies: resource requirements, data to be synced, setup commands, and the task commands.

Once written in this unified interface (YAML or Python API), the task can be launched on any available cloud. This avoids vendor lock-in, and allows easily moving jobs to a different provider.

Paste the following into a file my_task.yaml:

resources:
  accelerators: A100:8  # 8x NVIDIA A100 GPU

num_nodes: 1  # Number of VMs to launch

# Working directory (optional) containing the project codebase.
# Its contents are synced to ~/sky_workdir/ on the cluster.
workdir: ~/torch_examples

# Commands to be run before executing the job.
# Typical use: pip install -r requirements.txt, git clone, etc.
setup: |
  pip install "torch<2.2" torchvision --index-url https://download.pytorch.org/whl/cu121

# Commands to run as a job.
# Typical use: launch the main program.
run: |
  cd mnist
  python main.py --epochs 1

Prepare the workdir by cloning:

`1`	`git clone https://github.com/pytorch/examples.git ~/torch_examples`

Launch with sky launch (note: access to GPU instances is needed for this example):

`1`	`sky launch my_task.yaml`

SkyPilot then performs the heavy-lifting for you, including:

Find the lowest priced VM instance type across different clouds
Provision the VM, with auto-failover if the cloud returned capacity errors
Sync the local workdir to the VM
Run the task’s setup commands to prepare the VM for running the task
Run the task’s run commands

See Quickstart to get started with SkyPilot.

Runnable examples

See SkyPilot examples that cover: development, training, serving, LLM models, AI apps, and common frameworks.

Latest featured examples:

Task	Examples
Training	PyTorch, DeepSpeed, Finetune Llama 3, NeMo, Ray, Unsloth, Jax/TPU
Serving	vLLM, SGLang, Ollama
Models	DeepSeek-R1, Llama 3, CodeLlama, Qwen, Mixtral
AI apps	RAG, vector databases (ChromaDB, CLIP)
Common frameworks	Airflow, Jupyter

Source files and more examples can be found in llm/ and examples/.

More information

To learn more, see SkyPilot Overview, SkyPilot docs, and SkyPilot blog.

Case studies and integrations: Community Spotlights

Follow updates:

Read the research:

SkyPilot paper and talk (NSDI 2023)
Sky Computing whitepaper
Sky Computing vision paper (HotOS 2021)
SkyServe: AI serving across regions and clouds (EuroSys 2025)
Managed jobs spot instance policy (NSDI 2024)

SkyPilot was initially started at the Sky Computing Lab at UC Berkeley and has since gained many industry contributors. To read about the project’s origin and vision, see Concept: Sky Computing.

Questions and feedback

We are excited to hear your feedback:

For issues and feature requests, please open a GitHub issue.
For questions, please use GitHub Discussions.

For general discussions, join us on the SkyPilot Slack.

Contributing

We welcome all contributions to the project! See CONTRIBUTING for how to get involved.

Cloud Computing on Producthunt daily

puter

HeyPuter/puter

The Internet OS! Free, Open-Source, and Self-Hostable.

Puter

Getting Started

💻 Local Development

🐳 Docker

🐙 Docker Compose

Linux/macOS

Windows

🚀 Self-Hosting

☁️ Puter.com

System Requirements

Support

License

Translations

data-engineer-handbook

DataExpert-io/data-engineer-handbook

The Data Engineering Handbook

Getting started

Resources

Great list of over 25 books

Great list of over 10 communities to join:

Companies:

Data Engineering blogs of companies:

Data Engineering Whitepapers:

Social Media Accounts

YouTube

LinkedIn

X/Twitter

Instagram

TikTok

Great Podcasts

Great list of 20+ newsletters

Glossaries:

Design Patterns

Courses / Academies

Certifications Courses

skypilot

skypilot-org/skypilot

Run AI on Any Infra — Unified, Faster, Cheaper

Getting started

SkyPilot in 1 minute

Runnable examples

More information

Questions and feedback

Contributing