In today's episode we cover the following papers: A paper empirically studying a layer-pruning strategy for compressing large language models, showing that a substantial fraction of deeper layers can be removed with minimal performance degradation on question-answering tasks. Another paper introduces fully-fused multi-layer perceptrons (MLPs) that maximize data reuse to alleviate memory bandwidth bottlenecks, achieving up to 30x speedups over PyTorch for MLP-centric AI workloads on Intel GPUs. We also discuss the Octree-GS method that uses an octree hierarchical structure for efficient multi-resolution rendering of complex 3D scenes with Gaussian splatting primitives. Finally, we cover OPT2I, a framework leveraging large language models to iteratively optimize text prompts and improve their consistency with images generated by text-to-image models.

AI Pulse - Wednesday, March 27th 2024

In today's episode, we cover two research papers proposing novel techniques to enhance video generation and document understanding capabilities of AI models. The first paper presents AnimateDiff-Lightning, a lightning-fast model for high-quality video generation by applying progressive adversarial diffusion distillation and cross-model diffusion distillation techniques. The second paper introduces mPLUG-DocOwl 1.5, a unified approach for structure learning across multiple domains like documents, webpages, and images to improve OCR-free document understanding using components like H-Reducer and large datasets like DocStruct4M.

We then discuss a method called LLMLingua-2 for efficient task-agnostic prompt compression formulated as token classification and trained on a new extractive dataset. Next is the TnT-LLM framework that leverages large language models for automated text mining by generating interpretable taxonomies and using the models as data annotators. 

Finally, we cover a technique to transfer reasoning abilities from large language models to smaller vision-language models for improved chart question answering by utilizing techniques like continued pre-training, synthesizing rationale data, multi-task fine-tuning, and online arithmetic refinement.

AI Pulse - Wednesday, March 20th 2024

In today's episode, we cover the following papers: First, we discuss Google DeepMind's Gemma family of open language models, including the 7 billion and 2 billion parameter versions, which demonstrate strong performance across various language understanding, reasoning, and safety benchmarks. We then explore key findings and rules of thumb for continually pre-training large language models, such as re-warming and re-decaying learning rates, using replay data, and employing infinite learning rate schedules. Next, we overview the VLOGGER system for generating photorealistic videos of humans talking and moving based solely on audio or text input and a single image, highlighting its novel technical innovations and potential applications. We also summarize the TRUMANS dataset and a diffusion-based model for synthesizing realistic human-scene interactions, enabling controllable generation of human motions adhering to scene geometry and specified actions. Finally, we examine the SOTOPIA-π method for improving the social intelligence of language agents through interactive learning, behavior cloning from GPT-4, and self-reinforcement on positive examples rated by GPT-4, while discussing its limitations and the need for robust evaluation beyond LLM ratings.