Salesforce Will Hire No More Software Engineers in 2025

Salesforce will not be hiring any more software engineers in 2025 amid significant productivity boosts from AI, Marc Benioff has revealed.

“We’re not adding any more software engineers next year because we have increased the productivity this year with Agentforce *and with other AI technology that we’re using for engineering teams by more than 30% – to the point where our engineering velocity is incredible. I can’t believe what we’re achieving in engineering.

“And then, we will have less support engineers next year because we have an agentic layer. We will have more salespeople next year because we really need to explain to people exactly the value that we can achieve with AI. So, we will probably add another 1,000 to 2,000 salespeople in the short term.”*

...
2024.02.01: Open Source Sparse Autoencoders for all Residual Stream Layers of GPT2-Small
2024.02.06: Challenges in Mechanistically Interpreting Model Representations
2024.02.22: Do sparse autoencoders find "true features"?
2024.03.14: Sparse autoencoders find composed features in small toy models
2024.03.15: Improving SAE's by Sqrt()-ing L1 & Removing Lowest Activating Features
2024.03.29: SAE reconstruction errors are (empirically) pathological
2024.04.22: Mechanistic Interpretability for AI Safety A Review
2024.05.21: Mapping the Mind of a Large Language Mode
2024.06.13: The engineering challenges of scaling interpretability
2024.07.02: A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
2024.07.29: Detecting and Understanding Vulnerabilities in Language Models via Mechanistic Interpretability
2024.10.10: Bilinear MLPs enable weight-based mechanistic interpretability
2024.10.11: Explaining AI through mechanistic interpretability
2024.10.15: Mechanistic Permutability: Match Features Across Layers
2024.10.17: Using Dictionary Learning Features as Classifiers
2024.10.24: Probing Ranking LLMs: Mechanistic Interpretability in Information Retrieval
2024.10.25: Evaluating feature steering: A case study in mitigating social biases
2024.11.25: Adaptive Circuit Behavior and Generalization in Mechanistic Interpretability

Mechanistic Interpretability

I have prepared a list of papers on Mechanistical Interpretability. If you have good links on this topic, please share them in the comments.

• 2021: A Mathematical Framework for Transformer Circuits
• 2022.06.27: Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases
• 2022.09.14: Toy Models of Superposition
• 2022.09.24: In-context Learning and Induction Heads
• 2023.04.28: Towards Automated Circuit Discovery for Mechanistic Interpretability
• 2023.01.12: Progress measures for grokking via mechanistic interpretability
• 2023.05.24: Interpretability Dreams
• 2023.09: Sparse Autoencoders Find Highly Interpretable Model Directions
• 2023.10.25: Attention Lens: A Tool for Mechanistically Interpreting the Attention Head Information Retrieval Mechanism
• 2024.01.15: Sparse Autoencoders Work on Attention Layer Outputs
  ...

TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters

The paper introduces Tokenformer. The architecture leverages the attention mechanism to facilitate not only inter-token computations but also interactions between tokens and model parameters. The authors replace all linear projection layers in the Transformer with Pattention layers, allowing for efficient incremental scaling without the need for retraining from scratch.

Future work:
- Extending the Mixture-of-Experts Paradigm
- Advancing Parameter-Efficient Tuning
- Integrating Vision and Language Models
- Device-Cloud Collaboration
- Enhancing Model Interpretability

Code: https://github.com/Haiyang-W/TokenFormer

The Artificial Inflation (AI) of Artificial Intelligence (AI)—or AI^2 Bursts its Bubble, Bringing Down the Hype of the AI Threat

While some of the promises of AI have come true, and technology (like ChatGPT and its plugins) will continue to impress with its capabilities, AI-based technologies have largely failed to live up to the mountainous hype. In 2025, the authors expect the industry to pull back on the promises, investment, and hype of new AI capabilities and settle down into what is real versus marketing noise.