What AI Breakthroughs Happened This Week in AI News?

TL;DR

A wave of AI releases landed at once: models that generate live sports commentary in real time, tools that coordinate camera motion and character animation for filmmaking, and systems built for long or near-endless video. Microsoft added new Copilot agents, Perplexity brought a voice assistant to iPhone, and ByteDance released AI that controls a computer by reading screenshots.

Transcript

This week was pure madness in the AI world. We'll go over how AI can now commentate live sports in real time, how Alibaba solved one of the biggest problems in AI film making, and how SAND AI made it possible to generate longer videos without crashing your system. Microsoft introduced powerful new agents inside 365 Copilot. Perplexity launched its ... Read More

Key Insights

• Live CC is a model from the National University of Singapore that watches a game in real time, ingests the raw autocaption feed, and produces play-by-play commentary almost instantly, learning timing from messy, half-finished ASR fragments instead of tidy sentences.
• Alibaba's Uni3C coordinates camera movement and actor animation together by turning a depth map into a point cloud, steering the video diffusion model with a controller module, and welding both into one coordinate frame so feet stop sliding.
• Sand AI's MAGI generates long video by chopping the timeline into frame chunks and denoising them in an assembly-line fashion in parallel, using shortcut distillation and quantization so it can run on consumer GPUs instead of exploding VRAM.
• Sky Reels V2 claims near-infinite video using diffusion forcing, which keeps the last frames of each block overlapping with the next so context never evaporates, with weights released on Hugging Face under an Apache license.
• Microsoft's 365 Copilot Wave Two adds specialized agents, Researcher for multi-step web hunts and Analyst for spreadsheet data work, plus Copilot Search across tools like Slack and Google Drive and Copilot Notebooks that turns mixed documents into an audio summary.
• Perplexity's iPhone voice assistant can be bound to the action button or lock screen, lets users pick their preferred language model, and talks to apps like Spotify and Uber through Apple's shortcut hooks, though it is not fully hands-free.
• ByteDance open-sourced a model that operates computers by treating the screen as one image, predicting where to click, scroll, or type, trained on screen captures plus human and synthetic action traces, using pixels rather than fragile DOM trees.
• UC San Diego researchers ran a formal Turing test in which GPT-4.5, primed as a shy introvert persona, fooled most participants, though passing the test does not equal consciousness and suggests online identity checks need more than sounding human.

Questions & Answers

Q: How can AI now generate live sports commentary?

A research team at the National University of Singapore released Live CC, a model that watches a game in real time, ingests the raw autocaption feed, and produces fully formed play-by-play almost instantly. Rather than learning from tidy sentences, it trained on the messy, half-finished fragments an ASR system dumps out every few frames, and that noisy alignment taught the network timing, keeping latency very low. In a benchmark it beat much larger competitors, letting a single mid-range GPU handle live commentary.

Q: What problem in AI filmmaking did Alibaba's Uni3C solve?

Uni3C makes the virtual camera and the animated actor work together instead of tripping over each other. It takes a depth map, converts it into a quick point-cloud version of the scene, and hands that to a slim steering module called the PCD controller, which tells the main video diffusion model how to fly the virtual camera. At the same time it animates human bodies with skeletal bones, and both pieces are welded into one global coordinate frame so gravity points the same way and feet stop sliding.

Q: How does Sand AI's MAGI generate longer videos without crashing systems?

Traditional diffusion processes every frame together, which is why long videos overwhelm VRAM. MAGI instead chops the timeline into fixed frame chunks, denoises the first chunk, and while that is still warm begins the next, running several chunks in parallel like an assembly line of footage. Shortcut distillation squeezes the long sampling loop down to far fewer diffusion steps, and a quantized version can run on consumer graphics cards, making a short, high-resolution brand film feasible on accessible hardware.

Q: What is Sky Reels V2 and how does it achieve 'infinite' video?

Sky Reels V2, from Sky Work, makes a bold claim of infinite video generation. Its trick is diffusion forcing, which always keeps the last frames of one block overlapping with the next block so context never evaporates. Users can choose a synchronous mode for safer memory use or an asynchronous mode to live-stream frames while they are still cooking. A full high-resolution run consumes a large amount of memory but is doable on a workstation, and all the weights are released on Hugging Face under an Apache license.

Q: What new agents did Microsoft add to 365 Copilot?

Microsoft's 365 Copilot Wave Two introduces two specialized agents first: Researcher, which can run multi-step web hunts without opening a flood of tabs, and Analyst, which acts like a junior data scientist inside your spreadsheets. They live in a new agent store inside the Copilot app. Copilot Search now scours tools like Slack, Confluence, ServiceNow, and Google Drive and returns one blended answer with citations, while Copilot Notebooks lets you dump mixed documents into a single pane and request an audio podcast summary.

Q: How does Perplexity's iPhone voice assistant work?

Perplexity delivered its voice assistant to iOS, and while Apple still won't let you fully replace Siri, you can bind Perplexity to the action button or the lock screen. The assistant lets you pick whichever large language model you prefer among several options, and it speaks directly to apps like Spotify or Uber through Apple's shortcut hooks. It is not fully hands-free yet, but the video describes it as already feeling less error-prone than Siri for everyday requests.

Q: How does ByteDance's AI control a computer from screenshots?

ByteDance open-sourced a model that operates computers by looking at screenshots. It treats the screen as one giant image, predicts where to click, scroll, or type, and then sends mouse events through a small wrapper. Training involved a very large volume of screen captures plus human and synthetic action traces. On a synthetic desktop test bed it completes a substantial share of tasks and beats a comparable operator system. Because it uses pixels rather than fragile DOM trees, it could be a game-changer for robotic process automation.

Q: What did UC San Diego find about GPT-4.5 and the Turing test?

UC San Diego researchers ran a formal Turing test in which GPT-4.5 fooled a majority of participants, even beating some of the humans who were also in the test. They primed the model with a persona of a shy, internet-savvy introvert; when that persona was stripped away, it still tricked a meaningful share of people. The takeaway is that passing the Turing test does not equal consciousness, but it does mean online identity checks now need more than simply sounding human.

Summary & Key Takeaways

• The week opened with real-time media models: Singapore's Live CC generates instant sports play-by-play from noisy autocaptions, Alibaba's Uni3C unifies camera and actor motion for filmmaking, and ETH Zurich's Anom Portrait 3D turns a single sentence into a talking, blinking avatar ready for game engines.

• Long-form video generation advanced sharply. Sand AI's MAGI denoises frame chunks in a parallel assembly line to fit consumer GPUs, while Sky Reels V2 claims near-infinite video by overlapping frames between blocks so context persists, offering both safer and live-streaming modes with openly licensed weights.

• Assistants and agents spread across platforms: Microsoft embedded Researcher and Analyst agents in 365 Copilot, Perplexity shipped an iPhone voice assistant, ByteDance released screenshot-driven computer control, Baidu launched cheaper reasoning and multimodal models, and DeepMind warned that rare keywords can trigger hallucinations while YouTube tested AI clips in search.