​​Connected Papers: Explore in a visual graph

Connected papers is a unique, visual tool to help researchers and applied scientists find and explore papers relevant to their field of work.

Connected Papers are usefull to:
- Get a visual overview of a new academic field
- Create the bibliography to your thesis
- Discover the most relevant prior and derivative works

? Web-site: https://www.connectedpapers.com/

? BlogPost: https://bit.ly/3f32h8t
#⃣ #connectedpapers #research #arxiv

​​AI21 Labs Asks: How Much Does It Cost to Train NLP Models?

AI21 Labs Co-CEO, Stanford University Professor of Computer Science (emeritus), and AI Index initiator Yoav Shoham compared three different-sized Google BERT language models on the 15 GB Wikipedia and Book corpora, evaluating both the cost of a single training run and a typical, fully-loaded model cost.

The team estimated fully-loaded cost to include hyperparameter tuning and multiple runs for each setting:

- $2.5k — $50k (110 million parameter model)
- $10k — $200k (340 million parameter model)
- $80k — $1.6m (1.5 billion parameter model)

? Paper: https://arxiv.org/pdf/2004.08900.pdf

? BlogPost: https://bit.ly/2SllsBK
#⃣ #ai21 #nlp #gpu #bert #google

​​Jukebox: A Generative Model for Music - Opensourced by OpeanAI

Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. The model behind is VQ-VAE.

? Samples: https://jukebox.openai.com/
? Paper: https://cdn.openai.com/papers/jukebox.pdf
? Code: https://github.com/openai/jukebox/

? BlogPost: https://openai.com/blog/jukebox/
#⃣ #openai #jukebox #VQVAE

​​Determined: Deep Learning Training Platform

The platform aims to help deep learning teams train models more quickly, easily share GPU resources, and effectively collaborate.

Some of the benefits:
- high-performance distributed training
- intelligent hyperparameter optimization
- flexible GPU scheduling
- built-in experiment tracking, metrics storage, and visualization
- automatic fault tolerance for DL training jobs
- integrated support for TensorBoard and GPU-powered Jupyter notebooks

? Web site: https://determined.ai/developers/
?GitHub: https://github.com/determined-ai/determined
#⃣ #dl #determined #gpu #deeplearning

​​baikal - a graph-based functional API for building complex scikit-learn pipelines

baikal is a graph-based, functional API for building complex machine learning pipelines of objects that implement the scikit-learn API. It is mostly inspired on the excellent Keras API for Deep Learning, and borrows a few concepts from the TensorFlow framework and the (perhaps lesser known) graphkit package.

? GitHub: https://github.com/alegonz/baikal
#⃣ #sklearn #scikit #api #baikal

​​Free GPUs for ML/DL Projects from Gradient

Gradient Community Notebooks from Paperspace offers a free GPU you can use for ML/DL projects with Jupyter notebooks.

Main Advantages (over Colab):
- Fast storage comparing to Colab which uses GDrive;
- Gradient guarantees the entire session. Colab instances can be shutdown (preempted) in the middle of a session leading to potential loss of work;
- A large repository of ML templates;
- Ability to add more storage and higher-end dedicated GPUs from the same environment

?Gradient: https://gradient.paperspace.com/free-gpu
#hardware #gpu #freegpu #cloud

YouTube

Hello Gradient°

Effortless infrastructure for Machine Learning and Data Science https://www.paperspace.com/gradient

​​Solving classic unsupervised learning problems with deep neural networks

Discusses ideas from two recent papers: Learning gradient-based ICA by neurally estimating mutual information and Gradient-Based Training of Slow Feature Analysis and Spectral Embeddings
?Blogpost: http://bit.ly/33R99An
#⃣ #deeplearning #embeddings #unsupervisedlearning

​​NERD : Evolution of Discrete data with Reinforcement Learning

A toy project aimed at evolving sequence using an algorithm which is the combination of both Genetic Algorithm and Reinforcement Learning. The aim of the project was to evolve SMILES chemical molecules from scratch.

?GitHub: https://github.com/Gananath/NERD
?Blog https://gananath.github.io/nerd.html
#⃣ #rl #project #dl #nerd

​​Clash of Frameworks: PyTorch vs Tensorflow

A researcher at Cornell University compared references to TensorFlow and PyTorch in public sources over the past year. PyTorch is growing rapidly within the research community, while TensorFlow maintains an edge in industry, according to the report.

?Blogpost: http://bit.ly/2oDPsNB
#⃣ #research #analysis #pytorch #tf #tensorflow

​​Solving Rubik’s Cube with a Robotica Hand

OpenAI trained a pair of neural networks to solve the Rubik’s Cube with a human-like robot hand. The nets are trained entirely in simulation, using the same reinforcement learning code as OpenAI Five paired with a new technique called Automatic Domain Randomization (ADR).

?Paper: http://bit.ly/2OREuyp
?All Videos: http://bit.ly/35FRzRk

?Blogpost: https://openai.com/blog/solving-rubiks-cube/
#⃣#openai #rl #adr