Links for 2024-06-08

AI:

1. Eric Schmidt poached talent from Apple, SpaceX, and Google to create AI military drones for Ukraine. https://www.forbes.com/sites/sarahemerson/2024/06/06/eric-schmidt-is-secretly-testing-ai-military-drones-in-a-wealthy-silicon-valley-suburb/ [No paywall: https://archive.is/lkr04]

2. Scott Aaronson recommends Leopold Aschenbrenner's essay: "With unusual clarity, concreteness, and seriousness...Leopold sets out his vision of how AI is going to transform civilization over the next 5-10 years." https://scottaaronson.blog/?p=8047

3. Will jailbreaking soon be a solved issue? “We introduce Short Circuiting: the first alignment technique that is adversarially robust. Unlike adversarial training which takes days, short circuits can be inserted in under 20 minutes on a GPU. Unlike input/output filters, short circuited models are deployed as normal models with no additional inference cost.” https://arxiv.org/abs/2406.04313

4. Will We Run Out of Data? Limits of LLM Scaling Based on Human-Generated Data https://epochai.org/blog/will-we-run-out-of-data-limits-of-llm-scaling-based-on-human-generated-data

5. Self-Improving Robust Preference Optimization — “…we derive a practical, but mathematically principled offline algorithm to explicitly teach a model to self-improve and be robust to the choice of the eval task at the same-time!” https://arxiv.org/abs/2406.01660

6. MatMul-free LLMs: Proposes an implementation that eliminates matrix multiplication operations from LLMs while maintaining performance at billion-parameter scales. https://arxiv.org/abs/2406.02528

7. Grokfast: significantly reduces training iterations, accelerating the grokking process by 50 times in machine learning models. https://arxiv.org/abs/2405.20233

8. Buffer of Thoughts: Significant performance improvements over previous SOTA methods: 11% on Game of 24, 20% on Geometric Shapes and 51% on Checkmate-in-One. https://github.com/YangLing0818/buffer-of-thought-llm

9. BitsFusion: Compresses the UNet of Stable Diffusion v1.5 (1.72 GB, FP16) into 1.99 bits (219 MB), achieving a 7.9X compression ratio and even better performance. https://snap-research.github.io/BitsFusion/

10. YOLOv10, a powerful real-time object detection model, reduces latency by 46% and parameter count by 25% compared to its predecessor. https://github.com/THU-MIG/yolov10

11. σ-GPTs: A New Approach to Autoregressive Models https://arxiv.org/abs/2404.09562

12. Google releases new tool to automate Python code optimization. https://labs.google.com/code/transformer

Miscellaneous:

1. “Using the strategy game Civilization, this proof-of-concept study explores if strategy video games are indicative of managerial skills and, if so, of what managerial skills…We find that students who had high scores in the game had better skills related to problem-solving and organizing and planning than the students who had low scores.” https://link.springer.com/article/10.1007/s11846-020-00378-0

Links for 2024-06-07

AI:

1. OpenAI: “We're sharing progress toward understanding the neural activity of language models. We improved methods for training sparse autoencoders at scale, disentangling GPT-4’s internal representations into 16 million features—which often appear to correspond to understandable concepts.” https://openai.com/index/extracting-concepts-from-gpt-4/

2. Training LLM on structured data, like code, creates reasoning circuits applicable to a broad set of tasks. https://arxiv.org/abs/2406.02128

3. Robots are suddenly getting cleverer. What’s changed? On why multi-modal foundation models have resulted in a rapid & significant improvement in robots. https://www.economist.com/science-and-technology/2024/06/05/robots-are-suddenly-getting-cleverer-whats-changed [no paywall: https://archive.is/8cRzh]

4. “…the same amount of characters in the code are now completed with AI-based assistance as are manually typed by developers.” https://research.google/blog/ai-in-software-engineering-at-google-progress-and-the-path-ahead/

5. Google presents Open-Endedness is Essential for Artificial Superhuman Intelligence https://arxiv.org/abs/2406.04268

6. LSM1-MS2: A Foundation Model for MS/MS, Encompassing Chemical Property Predictions, Search and de novo Generation https://chemrxiv.org/engage/chemrxiv/article-details/6660add5418a5379b02860e8

7. Block Transformer: Global-to-Local Language Modeling for Fast Inference https://arxiv.org/abs/2406.02657

8. The Geometry of Concepts in LLMs https://arxiv.org/abs/2406.01506

9. “Comedy is a deeply human activity, requiring understanding of identity, social context, and audience. I interviewed comedy writers from around the world about using LLMs, what models can do, and where they fall short.” https://arxiv.org/abs/2405.20956

10. Anthropic launches a hands-on course on tool usage with AI agents. https://github.com/anthropics/courses/tree/master/ToolUse

Miscellaneous:

1. Microbes were engineered to "focus light into photonic nanojets." A sea-sponge enzyme, called silicatein, was fused to an E. coli protein that sits on the cell surface, called OmpA. The silicatein mineralized a polysilicate shell around the microbe that scatters beams of light. https://www.biorxiv.org/content/10.1101/2024.06.03.597164v1

2. “Why do colliding blocks compute pi? I'm still astounded this is true” https://www.youtube.com/shorts/P11ykXwx4-k

3. Computer science topics that might be relevant over the next decade?https://www.localcharts.org/t/topics-in-computer-science-i-think-are-important-over-the-next-decade/24202

4. “There are about 50 million people with Alzheimer's disease. This fraud resulted in 16 years of misdirected research before it was discovered. Depending on how much that delays finding treatments, this fraud might cause more suffering than any ongoing war.” https://www.science.org/content/article/researchers-plan-retract-landmark-alzheimers-paper-containing-doctored-images [no paywall: https://archive.is/Gh7bf]

https://www.youtube.com/watch?v=7WHbImUFM3c

YouTube

Nick Land - Kali Wave | Philosophy Hip-Hop Ep.24

*Available on all streaming platforms* Patreon: https://www.patreon.com/philomusic Instagram: https://www.instagram.com/artinsalimi666/ Discord: https://discord.gg/sFTRQWy696 - Lyrics: Darkness in my soul Makes me want to change it See Systems of…

In a dark, eerie badass future lives a cyber-gothic character. Its just a dream, spectacular backdrops, brilliant colors, in the style of cyber-gothic

Midjourney Art