obscurity

why work on something that’s not AGI?

after spending a few years in san francisco i became opposed to the concept of the american suburb. i liked that in the city you always had five friends who were at most a twenty-minute walk away, whereas in the suburbs you usually had none. i liked that convenience stores in cities were actually convenient to get to, and the houses didn’t all look the same. i liked that urban land was scarce so every park was painstakingly curated: usually situated on top of a hill, the space proportioned intentionally for sports / dogs / children / pedestrians, with cute plants and trees that accentuated the views, a far cry from the parks i’d grown up with which were usually just a flat patch of grass with an arbitrarily-placed playground

on some subconscious level i probably also enjoyed the idea of proximity to relevance. at this point i have friends at most major AI companies, which is an unusual feeling - you begin attaching human faces to every piece of news that comes out, and the development of technology becomes a much more engaging story. a similar process occurs in the reverse direction as well: when i first started working at Transluce, there were a number of previously uninterested people (mostly other people working in AI safety) who suddenly wanted to talk to me, and a few of those conversations evolved into legitimate non-work friendships

certain people will tell you that san francisco is the most important city in the world, for AI-related reasons, and anyone trying to accomplish anything of substance should move here. these are approximately the same people who will tell you that bringing inexpensive full-body medical scans to the public is irrelevant because that’s just a distraction from AI progress, or that academic researchers are behind the times and fraudulent, or any number of other claims that boil down to how only a few thousand people in the world are doing meaningful work and the closer your work is to the development of AGI the better. i think if you live in the city and work in AI this variant of exceptionalism begins shaping your worldview in subtle ways, even if you don’t explicitly subscribe to it. my friends sometimes find it difficult to distinguish the story of technological progress from the stories of their lives

all this is to say that i had mixed feelings when i began my current arrangement of spending most of my weekdays in palo alto working with stanford students while theoretically still living in san francisco. i was (and still am) skeptical of academic incentives, which seem to value papers and grants over doing useful work, and i was definitely not thrilled to be in the south bay (a few years ago i spent a summer in mountain view and was bored out of my mind). i knew i needed a change in environment, but it wasn’t clear if this would be the one that made me happy

---

one of the first things you realize when you join the salt lab is that people there take AGI quite seriously. the stereotype of the academic who remains in university because they believe AI progress will stall does not seem true here; stanford is too close to silicon valley for that to be the case. it might be more accurate to say that people understand the trajectory of AI and don’t feel the need to extract short-term profits from it. the first student i spoke with told me achieving AGI is not a very interesting goal to me. i want to figure out what will be useful for kids to learn after we have AGI (probably not technical skills; maybe some games like poker) and how to more effectively teach it to them

the conversation reminded me of my favorite scene from frieren, where frieren’s master tells her to avoid pursuing fame as it will only distract her from the goal of defeating the demon king:

the question of whether to pursue instrumental goals is always a difficult one. in frieren’s case, you could argue that fame would help her defeat the demon king by allowing her to recruit a better team (if you watch the show this argument is obviously incorrect). in the case of the student i mentioned, you could argue that working at a frontier AI company would give him the wealth and connections to better research the future of education afterwards (i think it’s ambiguous whether this argument is correct or not). in the case of elon musk, it’s probably true that he really did need to start paypal to amass the capital necessary to build rockets. rejecting instrumental objectives requires some belief in your future self’s ability to figure things out at a scale commensurate with your long-term goals

in my case, it’s still unclear what the exact long-term goal is, but i think it’s something along the lines of crafting stories that deeply resonate with many people, and the medium is probably some mixture of writing and technology. my experience with writing has been that frieren’s master is exactly correct - there is so much safety in obscurity (i’ve only attained virality a handful of times and each one derailed my life for several weeks), attention merely distracts from the work of improving as a writer, and one’s audience grows naturally as output and taste improve anyway. with technology, instrumentality is not as simple because distribution and user data really are useful but i’m hopeful that if you build something worth sharing it will receive the circulation it deserves. and perhaps as a result i am not in a hurry to leave obscurity behind

---

now that some time has passed i can say that i’m very grateful to be spending a few months in palo alto. it’s quite nice to have the freedom of a grad student without most of the obligations. it’s also been helpful to finally chat with people familiar with some of the social technology topics i’m interested in, and to get perspectives distinct from the ones that dominate san francisco

i started this post while sitting on a church bench i encountered on the hourlong walk from stanford to a friend’s house. i had to travel through a “park” consisting of a barren grassy rectangle with two soccer goals, and then an obnoxiously large parking lot, and then i paused because it was golden hour and i heard a choir rehearsing in the chapel. it was such an inefficient and thoughtless use of land, but also oddly idyllic: i realized that for the first time in months i had the space to think freely and do the work i wanted to do. i realized that, actually, the tranquility was pleasant and spending some time away from everyone in the city was probably good for me. i realized i could finally breathe again. all this time i thought i hated the emptiness of the suburbs i grew up in but maybe what really bothered me was the feeling of being a child, unable to choose