EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·Sep 3·5 min read
4.5B Posts Scraped from TikTok
TL;DR
A new scraping method for TikTok's private API has enabled the collection of 4.5 billion posts, yielding extensive data on videos, creators, and interactions. This dataset is now available for public use, offering insights into TikTok's ecosystem.
✦ Why It Matters
Start analyzing TikTok trends and user engagement by utilizing the newly available dataset for your projects.
Key Takeaways
- Explore TikTok's private API for richer data access.
- Utilize the dataset for machine learning model training.
- Implement robust error handling for API requests.
- Consider device registration and activation for successful scraping.
How It Works
The scraping system registers devices with TikTok to obtain unique device IDs and signatures. It then makes requests to the private API, ensuring that all parameters are correctly formatted and that the TLS handshake mimics a real device.
This method allows for the collection of extensive data, including video statistics and creator interactions.