EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Lobsters·Aug 17·5 min read
Self Extracting Tar Files
TL;DR
Self-extracting tar files can embed data in shell scripts, but recent methods have led to segmentation faults when handling files over 2GB. A new approach using base64 encoding maintains script integrity while addressing these issues.
✦ Why It Matters
Consider using base64 encoding for embedded data in shell scripts to avoid segmentation faults.
Key Takeaways
- Avoid segmentation faults by using fixed line removal in scripts.
- Use tar to package files without base64 encoding for large data.
- Ensure embedded data does not conflict with EOF markers.
- Test self-extracting scripts with files larger than 2GiB.
How It Works
The final method uses a shell script that removes the first three lines, allowing all subsequent data to be treated as embedded content. This avoids potential conflicts with markers and ensures reliable extraction of large files.