For the last few years, the Annual Review has followed the same three-stage template: 1. A couple of weeks of nominating posts for consideration in the review. 2. A month of writing reviews and critiques of posts. 3. Two final weeks voting on which posts were ultimately most valuable.
For the last few years, the Annual Review has followed the same three-stage template: 1. A couple of weeks of nominating posts for consideration in the review. 2. A month of writing reviews and critiques of posts. 3. Two final weeks voting on which posts were ultimately most valuable.
Here's a table of all the posts by year that were part of this process.
For the last few years, the Annual Review has followed the same three-stage template: 1. A couple of weeks of nominating posts for consideration in the review. 2. A month of writing reviews and critiques of posts. 3. Two final weeks voting on which posts were ultimately most valuable.
For the last few years, the Annual Review has followed the same three-stage template: 1. A couple of weeks of nominating posts for consideration in the review. 2. A month of writing reviews and critiques of posts. 3. Two final weeks voting on which posts were ultimately most valuable.
For the last few years, the Annual Review has followed the same three-stage template: 1. A couple of weeks of nominating posts for consideration in the review. 2. A month of writing reviews and critiques of posts. 3. Two final weeks voting on which posts were ultimately most valuable.
The exact process of the Annual Review is not locked in stone and wethe LessWrong team might decide to change it dramatically in the future. That said, for the past five years it has followed approximately the same basic format:
For the last few years, the Annual Review has followed the same three-stage template: 1. A couple of weeks of nominating posts for consideration in the review. 2. A month of writing reviews and critiques of posts. 3. Two final weeks voting on which posts were ultimately most valuable.
The LessWrong Annual Review is a yearly event where posts from a previous year are nominated, reviewed, and voted on.central mechanism for reviewing the site's content, it's kind of like our site's peer review process.
On LessWrong we're trying to makehas the goal of making intellectual progress on problemsimportant problems. To make progress, you gotta examine your community's outputs not only when they're first published, but also once enough time has passed to see whether they continued to provide value after initial hype fades and flaws have had time to surface.
So far, the bulk of the LessWrong Review happens in the winter season, over December and January. Batched yearly, we review the posts from the previous year. 2019's post get reviewed at the end of 2020 (i.e. every post in each review batch is 1-2 years old).
For two months, the community nominates, writes reviews, and votes on the most valuable posts. Reviewers ask questions like:
Does this post continue to affect my thinking or actions even 12+ months later?
Do its claims still seem valuable or interesting even now?
Was it invalidated by further work or other criticisms that matter, but itcame up?
Which follow-up work do I still wish someone would do?
The community answering these questions leads to benefits like:
We create broad knowledge about which ideas and results turned out to be robust and future work can be hardbuilt upon (and which ideas failed to tell whether we're making progress. achieve that).
Out of the thousands of posts published each year, we can direct the attention of future readers to the most valuable content.
e.g. via the creation of Sequences, [e]books, etc.
We can better reward and incentivize the thinkers whose contributions were most valuable.
You can see the top posts from previous years on the Best of LessWrong page. You can see the individual results page for each Annual Review so far: 2018, 2019, 2020, 2021.
[New!] The LessWrong team is currently experimenting with ways to give the Annual Review year-long salience, i.e. by giving people the ability to bet in prediction markets about which posts will be rated highly in the Review, and by giving Review "winners" prominent treatment (e.g. cool art on their post pages).
For the last few years, the Annual Review has followed the same three-stage template: 1. A couple of weeks of nominating posts for consideration in the review. 2. A month of writing reviews and critiques of posts. 3. Two final weeks voting on which posts were ultimately most valuable.
What is the Review process?
The exact process of the Annual Review is the once-yearly Schelling timenot locked in stone and we might decide to reflect, debate, discuss and vote. It's a poke to say "remember that idea you and everyone else were obsessed with in 20xx? Didchange
On LessWrong we're trying to make intellectual progress on problems that matter, but it can be hard to tell whether we're making progress. The Annual Review is the once-yearly Schelling time to reflect, debate, discuss and vote. It's a poke to say "remember that idea you and everyone else were obsessed with in 20xx? Did it continue to matter or was it just a fad?" It's also the time when we can determine which contributions most affected people in the long-term, and reward those authors and inspire others to follow suit. I think it's pretty great that when an author who invested hours into a post and greatly benefits the community, the community eventually recognizes and rewards them in some fashion.
Preliminary Voting Phase: Site members start identifying posts especially worthy of consideration in the review. Posts with 2 preliminary votes move into the Discussion Phase.
Discussion Phase: Site members begin reviewing and debating posts in earnest. This discussion will inform members on how to cast their final votes. Posts that receive at least one written review move to the Final Voting phase.
Final Voting: Site members allocate their vote budget (quadratic voting) among nominated posts. The outcome of the final determines the Annual Review results.
1. A couple of weeks of nominating posts for consideration in the review.
2. A month of writing reviews and critiques of posts.
3. Two final weeks voting on which posts were ultimately most valuable.
VotingResults