00:02:28Model-wise double descent
00:04:39Sample-wise non-monotonicity
00:06:14Epoch-wise double descent
ممکن است صدای این قسمت خراب باشد.
How to fix it?
درباره این اپیزود
We show that the double descent phenomenon occurs in CNNs, ResNets, and transformers: performance first improves, then gets worse, and then improves again with increasing model size, data size, or training time. This effect is often avoided through careful regularization. While this behavior appears to be fairly universal, we don’t yet fully understand why it happens, and view further study of this phenomenon as an important research direction.
Source:
https://openai.com/research/deep-double-descent
Narrated for AI Safety Fundamentals by Perrin Walker of TYPE III AUDIO.
---
A podcast by BlueDot Impact.
Learn more on the AI Safety Fundamentals website.
انگلیسی
انگلستان
یادداشت ها را نشان دهید 🔗
جستجوی اپیزودهای گذشته
اپیزودهای قبلی AI Safety Fundamentals: Alignment را جستجو کن.
قسمت های دیگر در این پادکست
Chain-of-thought prompting has demonstrated remarkable performance on various natural language reasoning tasks. However, it tends to perform poorly on tasks which requires solving problems harder than the exemplars shown in the prompts. To overcome this challenge of easy-to-hard generalization, we …
This post is about language model scaling laws, specifically the laws derived in the DeepMind paper that introduced Chinchilla. The paper came out a few months ago, and has been discussed a lot, but some of its implications deserve more explicit notice in my opinion. In particular: Data, not size, …
(Sections 3.1-3.4, 6.1-6.2, and 7.1-7.5)
Suppose we someday build an Artificial General Intelligence algorithm using similar principles of learning and cognition as the human brain. How would we use such an algorithm safely?
I will argue that this is an open technical problem, and my goal in this …
In this post, we’ll present ARC’s approach to an open problem we think is central to aligning powerful machine learning (ML) systems:
Suppose we train a model to predict what the future will look like according to cameras and other sensors. We then use planning algorithms to find a sequence of ac…
سلب مسئولیت: پادکست و آثار هنری تعبیه شده در این صفحه متعلق به BlueDot Impact است که متعلق به صاحب آن است و به Listen Notes، Inc وابسته یا تایید نشده است.
ویرایش
از کمک شما برای بروز نگهداشتن پایگاهدادههای پادکست سپاسگزاریم
