CORALINE: Hello and welcome to Episode 165 of America's favorite tech podcast, Greater Than Code. My name is Coraline Ada Ehmke. I will be one of your hosts today. And I'd like to introduce another of our imminent hosts that we have on the program today, my dear friend, Rein Henrichs.
REIN: That's a lot of episodes. I am here with my dear friend, Jamey Hampton.
JAMEY: Thanks. I was also thinking about how that was like a lot of episodes. I'm really pleased to be on the show today with all my friends and particularly with my friend who is our guest today, Penelope Phippen. Welcome to the show.
PENELOPE: Thank you. Thank you for having me.
JAMEY: Can you tell us about yourself because we didn't do a bio like we normally do.
PENELOPE: Yeah, sure. So my name is Penelope Phippen. My pronouns are she and her. I am a somewhat well-known Ruby developer and I am a Director of Ruby Central, the 501(c)(3) that runs RubyConf and RailsConf. For a very long time, I was a maintainer of the RSpec testing framework, but I recently retired from that to work on my new project Rubyfmt, which is a Ruby auto formater.
CORALINE: Penelope, the question we always open with, what is your superpower and how did you develop it?
PENELOPE: I think my most recently acquired superpower is extremely cursed knowledge of the Ruby programming language's grammar to the point where I can invent Ruby programs that most people don't expect to work, but they totally do. And I acquired it through building my Ruby auto formatter which led me to have this incredibly detailed knowledge of how the puzzle works.
CORALINE: What are you going to do with that? Are you going to use it for good or for evil or for awesome?
PENELOPE: [Laughs] Well, I guess good and awesome side is building a Ruby auto formatter. The evil side is that I can now win any trolling threads about the Ruby programming language anywhere on the internet.
JAMEY: I've literally already seen you do this at this knowledge. I was going to be like, "For evil, is posting about it on Twitter."
PENELOPE: Yeah. It is the most ridiculous review program you can come up or you're surprised this exists.
JAMEY: And then it's just like a bunch of letters and like semi-colons only.
PENELOPE: [Laughs] Yeah, basically.
JAMEY: I guess all words are just a bunch of letters and punctuation only.
PENELOPE: Yeah.
CORALINE: I can't wait for you to write a Ruby interpreter using white space. That would impress me.
PENELOPE: Sure, why not?
JAMEY: I want to congratulate you for the first person that I know of to come on the show and immediately describe their own super power as cursed.
[Laughter]
CORALINE: I think that is a first, yes.
PENELOPE: It is extremely cursed. Basically, I went into a cave and I got cursed. I came out the other side with this power that is both equally good and very, very evil.
REIN: Do you have like, so the Ruby grammar is written specified parse.y [Inaudible]. Do you have like, you know the matrix where they have like zeros and ones going down the screen? Do you have like snippets of parsed out why just like floating around in your head all the time?
PENELOPE: It's kind of like that but for Ruby files. I read a Ruby file and I can tell you exactly which parse nodes are being --
REIN: So, you see the matrix but for Ruby grammar.
PENELOPE: Yep.
REIN: So it is like the least useful version of [inaudible].
PENELOPE: Yeah, I think that's totally fair.
CORALINE: And then binding prize like think I need an exit.
[Laughter]
PENELOPE: Yeah, exactly.
REIN: I can see the matrix in this one very specific way that mostly annoys people.
PENELOPE: Yeah. I mean, the useful flip side of this is I can [inaudible] the existence of like pieces of Ruby grammars that I've never seen before and then write them and have them work the first time, which is how I discovered that any valid right hand side can go in a rescue clause in Ruby. So if you have rescue and then the FAT error you don't just have to put a local variable name there. You can literally put an entire clause definition in that position if you want to after the FAT error and then access a field on that clause definition. And that is completely legal in Ruby.
CORALINE: And I'm sure that would definitely make it through PR review.
PENELOPE: Yup. Absolutely. Why not?
CORALINE: Yeah.
REIN: It's like you showed me this on Twitter and I've been using Ruby for 15 years and I did not know that this was a thing because it's a stupid thing that shouldn't exist. And so, it's like syntax that is not actually like an object or an expression. It's like syntax that gets translated into [inaudible] calls like equals on whatever the thing is, which is sometimes a method call.
PENELOPE: Yeah. So if you want to go into the technical detail a little bit, I can absolutely expand on that.
REIN: I regretted that immediately.
[Laughter]
PENELOPE: Let me get my whiteboard out.
JAMEY: Oh, no!
REIN: I have made a huge tiny mistake.
PENELOPE: This is a Ruby interpreted design podcast now
REIN: I do have one more parse.y question for you though. And then I am happy to never talk about it again for the rest of my career. And that is when you were writing Rubyfmt, did you find any bugs in the Ruby grammar or its implementation?
PENELOPE: It depends what you mean by bug. Unfortunately, the simple answer is no. The complex answer is like 14 paragraphs of text about philosophically how Ruby parse trees should be constructed.
REIN: So, is this like the code is the spec kind of a deal? Like there is no spec for the Ruby grammar, so whatever parse.y does is the Ruby grammar?
PENELOPE: Yes, pretty much.
CORALINE: My understanding is that makes it very difficult for people working on things like JRuby or other implementations because there is no written standard for how Ruby is supposed to work. Is that still true?
PENELOPE: Yes. That's completely accurate. Philosophically, I guess the reason why I'm doing this, the way I am doing it is that there is no other implementation of the Ruby parser that you can just load into a program and execute. And so, JRuby has their own parser that is actually like a correct reimplantation of the Ruby parser but it's in Java so it's not like linkable. And then there's like the whitequark parser which is an attempt to reimplement the Ruby parser in C++. But that is not actually a fully accurate implementation of the Ruby parser, and they just gave up trying basically because they were like, "There is no spec for this. It's too hard to get it right." So, this is a good approximation for most people, but it's not truly accurate. So the reason why no one has done this before is that working with the Ruby grammar is just really, really difficult. And an essential side effect of this project is making that easier.
REIN: And you can't just rip parse.y out because it's like coupled to internals of JRuby.
PENELOPE: Yeah. It's funny you asked because I just looked into exactly what would you would need to do to be able to do this the other day. Basically when the CRuby interpreter boots, it does a bunch of global process level initialization things in C. So Ruby uses a stack based interpreter with its own object allocators, both memory-wise and in terms of the garbage collector. And all of that needs to be initialized before the Ruby parser can actually run. Because for example, when the Ruby parser parses a string, it allocates a Ruby string object. It doesn't just like slice out the string from memory. And so, you need like a partially booted Ruby virtual machine to be able to parse Ruby with Ruby's built in parser at all. And that's why it's so hard to extract out. And I'm at the very early phases all doing that work so that we can actually have, once and for all, a separate Ruby parser from the Ruby implementation.
REIN: Would CRuby ever switch to consume that parser?
PENELOPE: What do you mean?
REIN: If you would take parse.y out and you make it decoupled from the Ruby implementation, then could CRuby depend on your parser?
PENELOPE: Yes, it absolutely could. And some of the early conversations I've had with some of the Ruby folks suggest they might be open to accepting that. But this work is way too early to know whether or not that will happen.
REIN: Because that could help standardize the grammar across different implementations too.
PENELOPE: Yes, exactly. And in fact, for example in Sorbet, the Stripe type checker, Stripe actually decided that working with the Ruby parser in the virtual machine was going to take too much time for them. And so they actually forked the whitequark parser and basically did full re-implementation of the Ruby parser in C++. And the reason they basically gave me for that is they wanted an implementation, which they had full ownership of so that they could make changes they needed to frustrate Ruby in any case. And yeah, it's actually like a pretty interesting human story because the true underlying reason why the Ruby parser is like this is the backend, the one eight days Ruby interpreted Ruby Hoge just by walking the parse tree. And so a lot of the reasons it's like this are performance optimizations that Matz made in the really, really early days of Ruby in order to work. And so that was no external pressure for that to be a linkable version of the Ruby parser because the language just wasn't designed for this. And so, it's a really interesting side effect that because no one else was interested in the Ruby project in the early days of the development of the language, the parser has ended up being a thing that could only be used in the context of the Ruby virtual machine and not like on the outside. And so other teams are now building their own parsers for their own purposes.
REIN: So it's basically like if you want access to the Ruby AST, you have to first invent the universe, like at least 10 different teams have done that.
PENELOPE: Yes. I would say probably less than 10 but it's sort of close to that order of magnitude. I look at this situation, I'm like, "We're all trying to do the same thing here. Let's stop reinventing the wheel. I will just do the really, really hard work of pulling the parser out one time..."
REIN: [Inaudible] tribute.
PENELOPE: Yeah, just sort of attitude and only I am the kind of person where if you hand a really, really hard problem that seems tractable and no one is doing it because it's just -- the only reason people aren't doing it is because this is Hoge. But we know it's possible. We know we can do this with computers, so why not do it?
REIN: But it's also tedious.
PENELOPE: Maybe.
CORALINE: Penelope, the question that comes to mind for me is what problem are you trying to solve with this project?
PENELOPE: That's a great question. In order to talk about this, we need to talk a little bit about Rubocop, which is the other tool in the room. Rubocop is trying to sort of be the Swiss army knife, if you like, of working with Ruby source code. It's a formatter and a linter and it metrics and it does like bundler, Rails and [inaudible] cops, it does security. It does a whole bunch of stuff. And because of that, it doesn't do any one thing extremely well. As well as that, Rubocop has this sort of set of defaults, which I think based on my experience with all of the Rails teams I've ever met, no one is particularly happy with. But it also exposes every single thing that it does as a configuration option.
REIN: Every team I've ever worked with thinks some part of Rubocop's default are wrong, but they don't agree on which one.
PENELOPE: Right. And so, my observation has been that every single team comes up with that own Rubocop configuration and they sort of do it once and then they never ever look at it again. And this can have a couple of side effects. The first thing is Rubocop changes over time. And when it does, there may be new things you want to include or old things you want to get rid of. And the other thing is that your team and your organization changes over time too. And so the things you value about how your code works today might not be the things you value about how your code works in a year or two from now. And if you've got the sort of aging Rubocop configuration and people are taking it as sort of like a source of authority, that can become kind of problematic quickly.
REIN: And also there's a thing where it just like yells at you when you got stuff wrong.
PENELOPE: Yeah. And I guess it just sort of bounce off that. Rubocop can't actually automatically fix all of the things that it checks for, and frequently the error messages are not the most helpful about how to fix the problem that you encounter. I have had, since I started my project, about a number of more junior folks come up to me and be like, "I'm so excited for this because I keep arguing with my seniors to try and change the Rubocop file, and they weren't listening to me." And I'm like, "That sounds like you have a team dynamics problem. But if I can provide you a tool to solve your team dynamics problem, I will be more than happy to do that."
CORALINE: That is something that I have thought about quite a bit when you're establishing that initial Rubocop configuration. It's so impossible to do it by driving the consensus because there are just too many opinions in the room. So what happens is the loudest people or the people with the most political clout win the conversation and then that is encoded into this configuration file that, like you say, is long-lasting. And there is absolutely a power differential at play in how that gets created.
REIN: Let me put it this way. Do you know off-hand how many flags there are in Rubocop?
PENELOPE: I know it's strictly more than a hundred and strictly fewer than a thousand.
REIN: So what that means is there are more possible configurations of Rubocop than there are particles in the universe.
JAMEY: [Laughs]
PENELOPE: I guess, yes, that's certainly true. The thing is with this take, you could sort of reflect back onto me that if I'm making a Ruby auto formatter that has zero configuration flags, that I'm also silencing the least powerful people in the room. But what I'm actually doing is silencing everyone in the room. And the way I think about this is that designing specifically a code, ignoring all the other things that Rubocop can do, designing specifically a code formatter, the outputs Ruby source code that everyone can agree is consistently formatted and is something that could be called a correct formatting even if it's not everyone's favorite. Building that formatting is an extremely hard problem because there are so many little fiddly edge cases and when you move things on to new lines and how you do indentation and which constructs you use and that sort of thing. And I found very quickly that actually once you have built just like a functional auto formatter that can take source code in and spit source code out, there's at least as much work if not more in deciding how the source code actually looks. And I think that, to sort of come back to the point, Coraline, you were making, that the loudest voices like consensus is certainly the wrong way to make those kinds of very subtle and hard decisions because you actually just need to cook them over a long time in a single person's brain. It's not the kind of decision you can make quickly and rationally even through conversation. You need to look at lots of code, you need to format it, you need to see how it feels. You need to actually do a sort of more material exploration. And I think a lot of teams assume they can solve this in like a single two or three hour session and the problem is over, but actually everyone will end up hating it regardless.
CORALINE: The thing that I use Rubocop for, the only cases in which I use it are if I have a project that multiple people are going to be contributing to at skill levels that I cannot predict. And the main output, the two main things I want from it are a code base that is consistent, and I don't even care about the rules necessarily to a granular degree as long as there's consistency. But the other aspect of it, and this is like a human cost, is I don't want someone to be nitpicked in a PR review over syntax.
PENELOPE: Yeah. I completely agree with you. I think I sort of kind of have a hierarchy of nitpicking that I sort of came up with when I was thinking about this. The base level is you have no tool reviewing your code and formatting whatsoever. And so it's humans nitpicking humans. And what will happen there naturally just over time is the people at the top of the power dynamics will nitpick to death the people at the bottom of the power dynamics. I know a number of people. Betsy Haibel I think is a really good example of this of people who've just gone on [inaudible] talking about problem. And that's sort of the next level up. You kind of have the Rubocop universe where you have a configurable formatting tool where no one is nitpicking style preferences in a pull request because you either get a check mark or like a failure mark, but you can nitpick people's preferences at configuration time. So if someone wants to propose a change at configuration time, you get that same power dynamic except it's almost more ingrained because there's so much authority associated with the configuration of a tool like that. And what I've generally seen, and this comes more from my work on Go than it does my work on Ruby where they have gofmt and golint, which are literally not configurable. The language authors decide 'this is what code looks like, end of solution' is like there's no one you're interacting with on a daily basis where that power dynamic can apply. Now, the sort of up-level of that problem is then you versus language author and that can be really, really hard in those communities. I'm not going to sit here and pretend the people who are at the top of the go community have the best reputations for dealing with underrepresented minorities in tech. But I think assuming you have a good community like the Ruby community has, I think having those defaults be evaluated as a community basis by someone who is well known and well established is a really good way to solve this problem.
CORALINE: Are you collaborating with anyone on the decisions that you're making, Penelope?
PENELOPE: I very much intend to. Straight up right now, Rubyfmt is not in a place where it's ready for all but sort of like the simplest of decisions. An example of a style decision that has been made is the one I presented in my RubyConf talk, which is like to do with relative indentation to the parent of if statements and string embeds and case statements and stuff. It's kind of hard to explain that in an audio form. And to some degree, I'm working with Justin Searls to iron out a bunch of these cases. Sidebar, part of my motivation for doing this project when I started it was Justin sort of started work on his project which is called Standard. Standard is basically trying to be wrapped Rubocop and just decides the good configuration flags and then remove the ability to configure a Rubocop. I was like, "That's cool," but I really want a go format like tool where I can have it in my editor and it saves so fast that you can't even notice the formatting is happening, and let's at least make sure these things are compatible. And so, Standard -- I completely derailed my train of thought.
REIN: Standard is configuring Rubocop and making it so that there are no flags but you want it to be fast enough to work on the same.
PENELOPE:
Yeah, which you literally can't do with -- actually any tool that requires bundler to be present. Booting bundler takes somewhere in the order of 400 milliseconds and that's just too slow for interactivity.
REIN: So you're doing this in C.
PENELOPE: I am doing this somewhat in C and mostly in Rust. I have programmed plenty of C in my life enough to know that I hate debugging memory safety issues and the Rust programming language gives you native performance without having to debug memory safety issues.
REIN: If only there were some way to just not have to debug memory safety issues.
CORALINE: So Penelope, what are the major challenges your facing better not at the code level?
PENELOPE: That's a really interesting question. I think one of the biggest ones is that like the -- well, I guess this is really at the code level in a sense -- but the Ruby programming language has no implementation documentation. So I'm having to -- I don't know if this is what you were looking for. Actually I'll kind of say mostly I'm working on my own right now and just writing a lot of code and having no communication overhead and just building a thing is really nice. But yeah, I don't know that I'm actually facing any sort of like non-code level challenges with this project right now. Just time and motivation really. This is something I'm doing in my spare time not as part of my job.
CORALINE: So you haven't gotten any pushback or anything like that from the community?
PENELOPE: No. I would say if anything, the reception to this has been overwhelmingly positive. No one seems to think this is a bad idea, which is actually super encouraging. I guess one thing I did have, perhaps this wasn't a challenge so much as it was a misunderstanding is a lot of folks were like, "How do I install the gem?" And I'm like, "This is not a gem and it will never be a gem," which is like, it's the community standard way of delivering software. But the reason I'm pushing against that is it's kind of like a lot of people were expecting this to be distributed as a gem, which is like the Ruby standard way of distributing packages and it's not a gem. And the reason for that is like a whole host of implementation reasons the aren't worth getting into. But it was really interesting to me to be building this Ruby thing and it being a bit nonstandard, and people having such this overwhelming expectation of how it would be distributed. And me being like, "Nope, I have my reasons. Please believe me these are good reasons to not do it this way. It will be fine, I promise." And actually it's really interesting because the Sorbet team ran into the exact same problem. All of the code for Sorbet, it's a native C++ program that they could just compile to an executable on both Mac and Linux. And they also created a gem, even though there's no reason, there's literally no technical reason why that needs to be the case. They said that they did that just so that people would have it in the normal workflows and they could instruct people to upgrade and do a bunch of those distribution things in a way that they're used to doing. And it was very interesting to me to be like, "This is not the same as the things you are used to receiving in the Ruby community," and so I'm distributing it in a different way. And there was quite a lot of confusion around that.
CORALINE: Penelope, how will that actually work when you are finished with Rubyfmt then other people will have to take that and adapt it somehow to a variety of editors using a plugin architecture? How is that exactly going to work?
PENELOPE: There will be editor plugins for all the major editors. Actually, it's really funny. People started building editor plugins long before it was ready for any kind of daily usage. So I will say, and this sort of comes back to the response that I've gotten subject is like I was overwhelmed by how positive the response was. There are I think vscode and Atom plugins for Rubyfmt already that I did not write. I don't use vscode or Atom. I also have no idea how good those plugins are. And so, I was just like, "Wow, okay. This crashes on a certain percentage of Ruby files today, but good for you for sticking with it." I mean, the core of it will just be a binary, like a Linux or Mac OSX binary that you download. There will be a website where you can pull updates. I will probably ship an update flag in the binary as well just to download it or compile from code. But almost none, in fact probably by the time I am done, none of the code in Rubyfmt will be Ruby, and so it doesn't make sense to try and distribute it as a Ruby gem.
CORALINE: Yeah, that makes sense.
REIN: So, like OS package managers?
PENELOPE: Yeah, Brew or cURL piped to pseudo bash. You know, that old Chestnut.
CORALINE: What could possibly go wrong with that? So, Jamey, you had a question?
JAMEY: I do. I think it's really interesting that Coraline asked what are the non code challenges and you answered like. "I haven't really had non code challenges other than finding the time to work on this personal project," or whatever which I think is one of the biggest non code challenges that any of us face ever. And so, I want to ask about that. I want to ask about like how have you carved out time for this project and how do you motivate yourself to work on something like this. I mean, you've done lots of projects. How do you motivate yourself to work on these things in your spare time outside of all of the other things that you're doing?
PENELOPE: That's a great question. It's a little hard for me to pinpoint exactly what it is. Part of this is like Rubyfmt is genuinely, I think, the hardest thing I have ever done with a computer by some distance, and I'm loving bathing in the technical complexity. So, some of it is that for sure. I think having a thing that is truly mine to just sort of pull on has been great. When I was working on RSpec, I was one of several maintainers, and so I felt some ownership but it's not the same thing as having complete and total ownership, which has been huge for me frankly. It really depends. If you look at the GitHub [inaudible], there will be entire months where I don't do anything and it's fine. And then I'll have like months where I write literally thousands of lines of Rust code. And so my personal motivation definitely swings. I don't want to get super far into it, but I also sort of medically and socially transitioned over the last nine months and that has been part of the development work. There have been days where I've just been like, "Hormone's too overwhelming, I'm going to sleep now," instead of working on Rubyfmt. And so, it's really variable to be honest with you and a lot of it just depends on my mood and whether or not I feel like attacking the problem today, but it's a lot of fun.
REIN: Penelope, you were saying that you really haven't gotten pushback on your plans for Rubyfmt. I was wondering if you could maybe try to elaborate on what you attribute that to?
CORALINE: Penelope is a superstar and everyone wants to do what she says.
REIN: Yes, and what else?
[Laughter]
PENELOPE: If I were to guess, it feels so -- I mean, I can't tell what's in other people's heads. But I can tell you why I wanted to do this. I've been thinking about a tool of this form for close to five years now. I've been thinking about building something like this for actually quite some time. And the reason I never really got started is I didn't feel any sort of pressure or need because the Ruby community sort of had consensus on Rubocop and there was no other tooling being developed in this space. Literally, no one was doing anything. And then last year at RubyConf, Justin came along and was like, "I'm going to build Standard and here's why I think it's a good idea." And I was like, "Okay." His was somewhat personal story. I was like, "I can't live with the idea that we are all using Rubocop as like the final formatting thing." Because the way Rubocop interacts with the Ruby parser just can't do the level of detail I want a formatter to be able to do. This is also like my deep and personal need for perfection. I've been thinking about what a tool of this form would look like for, as I said, about five years now. It was really inspired by when I saw gofmt working. So when I first worked in the go programming language, I'm like -- if you've not ever worked in the go programming language, basically the out of the box editor tool chains all set up auto format on save and you save on your file just snaps into place. Like the code just becomes correct and it's this absolutely beautiful feeling. It's kind of like little dopamine hits every time you save a file. And I know folks in the go programming language who have sort of said like, "For me, if I'm coming to another programming language now, having a tool like this is absolutely table stakes." Because I don't know how folks are laying out their source code in those languages. And if there's a tool that does it for me, that's so much the better. One of my actually biggest people who's been cheering me on is a friend of mine. He's like a very experienced go developer who occasionally has to touch Ruby. And he's like, "I always get style nitpicks on my pull requests, please make them go away." And I was like, "I will absolutely do that." And so, I think if I were to sort of point out why the response has been so positive, it's that there actually isn't another tool in this class for Ruby today. Rubocop is close, but it's not doing the same. Or certainly, it's not doing what I am promising my tool will do when it's done. An open question is whether I can do that. I think I can. But the other thing I should say is it's not certain. And so partially, I think this is also a hype cycle and I've clearly just hit on a thing people really, really want that I have really, really wanted for a long time and now, I'm actually dragging myself over the hot coals of working with the Ruby parser to do it.
CORALINE: I think that work is greatly appreciated, Penelope. I know I'm personally very excited about this because I did work in go about a year ago and go is the language that makes me happy. I know it's really good at some stuff, but I just don't enjoy writing it. But gofmt was magic and it made learning the language so much easier.
PENELOPE: Yeah. And that's part of it as well. One of the things I observed with my working in go actually, and you've sort of hit the nail on the head is learning the language, is like the first time I would write some code, I would lay it out and fire a pull request up and [inaudible] fail CI for formatting reasons or linting reasons. Because they also have a standard linter built in. And I would slowly learn better how to write go code to the point where I was almost always doing what the formatter or the linter would want before I even ran. That as a teaching mechanism I think is really interesting and we don't have tooling like that in Ruby. And I'm sort of taking on the grand challenge of building, not just the formatting tool, I want to ideate and build that teaching tool in the future at some point as well.
REIN: I have maybe like a somewhat heretical view on this, which is that the fact that we're even writing source code in text files as sequences of characters that then need to be formatted is a weird and unfortunate historical accident.
PENELOPE: Yes.
REIN: That's it. That's the whole take.
PENELOPE: [Chuckles] Okay. Let me ask you a question. What would you have us do instead?
REIN: [Sighs] I want an ergonomic way to work directly with ASTs.
PENELOPE: Cool. Would you preserve the existing programming languages or --
REIN: No, they're all forced up against the wall. [Laughs]
PENELOPE: Okay. So, we would have to have a new programming language explicitly designed with an ergonomic AST.
REIN: I mean, it's not going to happen, but yes, that's the thing that would never happen.
PENELOPE: Cool. So Rein just reinvented Lisp.
[Laughter]
REIN: Funny thing about Lisp is that it was originally supposed to be the end language that they were going to write an ergonomic thing on top of.
PENELOPE: Yeah. I would put to you that if Ruby developers were forced to hand edit Ruby ASTs instead of Ruby source code files, there is no way to make that more ergonomic.
REIN: No, that would be terrible. But I think something can be done that's better. I just don't know what.
PENELOPE: You can theorize programming languages I think where that would be the case, but taking existing ones and trying to make that happen would be extremely painful. For example, the way C and C++ and Ruby and a bunch of other programming languages parse like, list like, or hash like structures is with tail recursion. And then you have to teach a whole generation of programmers how to do tail recursion.
REIN: I told you that was heretical.
JAMEY: [Inaudible] label your take. Thank you.
PENELOPE: I like this take. It's a take I have had in the past as well. I just think that the practical way we get that is unfathomably difficult and starts in CS research in universities and takes 50 years to leave that space.
REIN: There is a little bit of work that's being done with editors that actually sort of guide the code as you write it rather than waiting until you write a bunch of stuff and then save it. So, we'll put the cursor where it needs to be for the thing you're currently doing.
PENELOPE: Yup.
CORALINE: I believe I would hate that a lot. Penelope, what else do you want to talk about?
PENELOPE: One thing I think is potentially interesting. I don't have answers here. I have questions more than anything. So at the Ruby Central directors' retreat, right before RubyConf, we were talking a little bit about how we were lamenting the death of regional Ruby conferences. It really feels like they've all disappeared. I don't know if this is also happening in other programming languages that have been around for as long as we have or what's going on there. We had some sort of theories. But my question is, is having regional Ruby conferences a good thing in the first place and if it is, what can we do to bring them back?
CORALINE: I can say for a lot of people, regional Ruby conferences can launch their speaking career. There's only what, 35 slots for RubyConf.
PENELOPE: Sixty, including keynotes.
CORALINE: Yeah, a very limited number of slots available. And that conference is hard to get into. It took me five years to get accepted at RubyConf and I speak all of the time. So, regional conferences are how I got my start, and I think they're valuable for other people. I do miss them. We used to have so many five years ago and now there's just a handful. And I think some people interpret that as a reflection of the language itself. But I wonder, I guess like you were saying, is it an evolution of the community?
JAMEY: I do a lot of regional conferences, not a lot, but I tend to do regional conferences including ones that aren't like where I'm from because I'm jay speaking. But like DevOps days I think is a great example of a community that's still doing tons of regional conferences. And I think that what's really interesting for me going to them, particularly going to them in other places. And being an out-of-town speaker among attendees that are mainly local is kind of an interesting experience for me. But it gets me thinking of like -- I agree with what Coraline said about speakers getting their start, but I also think that it's like an accessibility thing for folks that can't afford to go to a conferences out of town. And so, like a conference comes to your town and you go, and I've noticed this in other places, but like Buffalo just had our first DevOps days. And I had been doing a couple of DevOps days conferences and it was kind of neat talking to like, I didn't know a ton of the people there because Buffalo doesn't have like a super tight knit tech community. But it was mostly local people and it was mostly local sponsors. And I was surprised by that and it was a cool experience. And so, I think that that is really valuable and that is something that I think is missing if we're congregating at these bigger events where mostly everyone is traveling.
PENELOPE: Yeah. So I really only know two, I have two like one person anecdotes about this. So, one is that a local conference organizer who's been doing it for a number of years told me they were struggling to get enough speakers to fill that conference and another is that organizers just burned out. And there wasn't a generation of organizers to replace them. I guess my working theory is that we are potentially responsible for creating a whole new generation of local conference organizers and making it. And here when I say we, I mean Ruby Central as the sort of position as the steward of the community, creating a new generation of conference organizers who can run those conferences. Because to me that seems to be the thing that dried out.
REIN: A thing that I've heard from a few people who've run or ran these regional conferences is that they weren't really trying to make money but it was hard to even breakeven.
PENELOPE: Yeah.
JAMEY: Yeah. I was also going to bring up money. I've been an organizer on a regional conference and I'm not one of the main organizers, so I don't deal really heavy in the sponsorship stuff. But I know that it's a worry and I know that we didn't have our conference this year because we didn't have the money to have it. And we kind of went, "Well, this is still important to us, so we're going to regroup and start earlier and start looking for sponsors for 2020." But in an ideal world, we wouldn't have had to like not have our conference in 2019.
PENELOPE: I think the thing that I have heard that's often really scary is many conferences don't breakeven, they make a small loss, but they only get really close to it right before the conference happens if they do breakeven or like make a small profit. And so right up to the deadline, you're looking at a huge financial shortfall. And for individuals, that can be extremely scary.
CORALINE: Is that something Ruby central could [inaudible].
PENELOPE: I don't know. We'd certainly have to review that.
REIN: So it sounds like the things that are needed for regional conferences to be sustainable are sponsorships so that they can not feel super stressed out about a financial disaster because they're trying to bring people together to talk about a programming language. Is that the main, or what are the components?
JAMEY: I think another thing that got mentioned was lack of speakers. And I think that as a speaker and someone who like, I actually really quite like to do small regional conferences in other places, but I can't afford to travel for conferences that can't pay a travel stipend to me. And I haven't been at a company that can help me out. And so, it's not that I don't want to do it and it's not that I'd begrudge them necessarily for like -- being in a situation that I know I've been in as an organizer where we don't have money, but it's just kind of unfortunate. I think if more money was there, that could alleviate some of these problems, even more of these problems than is immediately apparent potentially.
PENELOPE: Yeah. You were sort of arriving at this really interesting point, which is assume that was like financial oxygen such that any organizer knew that would be no financial burden on them whatsoever, would that change the dynamic?
JAMEY: I think it would.
PENELOPE: Yeah. And I think the answer is probably yes because I know when there's at least one regional conference that I'm thinking of, that I was about to say the name off but I'm not going to, that spun down essentially because it stopped being financially viable for them to run the conference. It's really interesting.
REIN: Do you think that the DevOps days regional conferences are sustained by having a lot of developer relations folks with people whose job it is to go to conferences speaking at them?
PENELOPE: I don't know, I wouldn't be surprised -- because it does a lot of money sloshing around in that ecosystem. It wouldn't surprise me at all if like the top -- you could probably get five to 10 reliable speakers just because that company will pay for any amount of travel. All the big players in DevOps have the travel budget and I guess that's less true and there are fewer dedicated Ruby developer evangelism teams with budget these days.
JAMEY: As someone who speaks at DevOps days conferences, again, I pretty much go to ones where they can pay me a travel stipend because I don't have that. But I think it's possible that the money that's going towards my travel could be being kind of bankrolled by other speakers who aren't asking for that financial assistance. And I don't know about that, if that makes sense.
REIN: So how do we get avocados to come to Ruby conferences?
JAMEY: I feel like what I would prefer over that -- I mean, I am a developer advocate, so I'm not like making fun of the avocado thing. But I've talked a little bit about organizing and I haven't said what conference and I will. I'm an organizer for Catskills Conf. And what we've tried to do, like particularly it takes place in the Hudson Valley. I'm not located in the Hudson Valley, but most of the organizers are. And they push really hard for local people to be speakers. We don't have a lot of speakers coming in from out of town. I'm coming in from Buffalo, which is a 5-hour drive. And I'm usually one of the farthest away people coming, including the speakers. And I think that it's been a good experience overall. I mean, obviously as I said, we're still struggling with sponsorships and stuff because I think that's just a thing that goes around. But it's been a great event and it's been a good perspective to have people from that community talking about tech. But like talking about tech in terms of that community has been, I think, rewarding for that group of people, if that makes sense. And I would love to get more people involved in speaking rather than making a point of trying to get the same people over and over in the way that you're describing.
PENELOPE: And I think at the very height of the Ruby conference regional days, that was a little bit more of that. I know I used to be able to do 10 to 15 new conferences in a year, without even having to try all that hard. And that was just because they were like, "Hell yeah, come give the exact same talk in 10 different cities." And I was like, "Cool." But there were loads of people doing that. There were people who I went to four or five or six conferences within the same year and also gave the same talk four or five or six times. And so, I do still wonder if it was like lack of growth of local talent and external talent sort of pushing that out a little bit.
REIN: I think it would be unfortunate if the only way to sustain these local conferences is also pushing out local speakers and things like that.
PENELOPE: Yeah.
REIN: Cool, super upbeat way to end the podcast.
[Laughter]
REIN: I really missed -- I love how diverse they were. The cities are all different obviously, but they were not all in big convention centers. A lot of the venues are cool and unique and just a whole bunch of people talking about stuff that you wouldn't get to see at RubyConf like Coraline was saying. So, I missed them.
PENELOPE: It was funny, I was just talking to Ernie Miller yesterday and he was lamenting that he feels like his favorite talk to give is not one he could give at a big conference because it's too risky. And I was like, "That's such an interesting way of putting it." And then he told me a little bit about the idea and I was like, "You're right, I would stop you from giving that talk at RubyConf because it would almost certainly go wrong in horrible, horrible ways," and there is no backup strategy for that. So yes, I agree with it. You got riskier, more interesting talks, less hermetic.
CORALINE: Less of the same people speaking over and over and over again.
PENELOPE: Yup.
CORALINE: I love Aaron Patterson, but he speaks at every conference.
REIN: [Inaudible] Being very funny though.
CORALINE: That's true. But there is a core population of speakers that speak at every major conference and I too miss the quirky, the local flavor, the up and coming people. I knew for me it was a career changer and I hate the fact that that is closed off to so many people now.
PENELOPE: And I think that career changer point is the one that's perhaps most important is I don't think I would be where I am today in my career without access to -- and I guess to bring it back, speaking at 10 to 15 conferences per year, that put my name in the ears of literally thousands of people that I would otherwise never have met.
REIN: Yeah. They changed a lot of people's lives, myself included.
PENELOPE: Oof.
REIN: Cool. It's even worse now.
PENELOPE: I'm sorry to have created a big sad [inaudible].
[Laughter]
JAMEY: I'll say something kind of a little bit less sad, which is that conferences and doing speaking has changed my life too, and I kind of am not old enough in the industry to remember some of the things that you are all talking about. By the time I started doing conferences, a lot of this was already on the decline and it's kind of sad to listen to you all talk about something that I kind of missed out on. But I think in other ways, I don't know, I'm having trouble with what I want to say because I'm not the expert, I guess. But I've had a pretty good experience at conferences and my experience has been probably getting better since I started a few years ago, in that more conferences seem like they care about new speakers and diverse lineups. And I hear some kind of this, "Oh, the good old days," with how conferences used to be and these smaller events and this. This isn't the first time I've heard that. But I've also heard a lot of horror stories about how things used to be and I think that the conference culture that I'm in now, I like it.
CORALINE: It definitely has changed. I mean, it was only five short years ago that we had raging debates about whether conferences should have codes of conduct.
PENELOPE: Yeah, that's true.
CORALINE: That was her real fight.
PENELOPE: Yeah. And like, the Ruby Central conferences, we provision gender neutral bathrooms and a childcare facility. It's now completely consistently. That certainly wasn't the case five years ago either.
REIN: Yeah. To build on this and maybe bring it back to a happier place, maybe if not like there was this prelapsarian time when the Ruby conferences were very good and that's all lost now. Maybe it's just that the industry is changing and we don't have those conferences anymore, but we do have other regional conferences in different parts of the industry, and there may be even better than what we had before.
CORALINE: Yeah. I mentioned something about that because I found that I have shifted from local Ruby conferences or regional Ruby conferences to regional Polyglot conferences. And I actually wonder if that might be better because for a long time, the Ruby world was pretty insular.
PENELOPE: I could see that.
REIN: For myself, I, like for a decade, considered myself a Rubyist. That was like the primary thing for me. And only in the last five years or so, I really started with say, "I'm an engineer who knows Ruby."
PENELOPE: Yeah.
CORALINE: Well, Penelope, it's been an absolute delight to talk to you. I'm so proud you came on the show.
PENELOPE: I'm glad. Thank you.
JAMEY: Penelope, I'm so glad that you came on the show. It's so wonderful to have you on and for us to all be able to chat and it was delightful. So, thank you so much and thank you to everyone for listening. If you want to continue with some of these conversations with us, you can always back us on Patreon and that will get you access to our Slack community. Everyone who's ever been on the show has access to this community and we have great conversations in there and we'd love to have you. Thank you.
REIN: That's a lot of episodes. I am here with my dear friend, Jamey Hampton.
JAMEY: Thanks. I was also thinking about how that was like a lot of episodes. I'm really pleased to be on the show today with all my friends and particularly with my friend who is our guest today, Penelope Phippen. Welcome to the show.
PENELOPE: Thank you. Thank you for having me.
JAMEY: Can you tell us about yourself because we didn't do a bio like we normally do.
PENELOPE: Yeah, sure. So my name is Penelope Phippen. My pronouns are she and her. I am a somewhat well-known Ruby developer and I am a Director of Ruby Central, the 501(c)(3) that runs RubyConf and RailsConf. For a very long time, I was a maintainer of the RSpec testing framework, but I recently retired from that to work on my new project Rubyfmt, which is a Ruby auto formater.
CORALINE: Penelope, the question we always open with, what is your superpower and how did you develop it?
PENELOPE: I think my most recently acquired superpower is extremely cursed knowledge of the Ruby programming language's grammar to the point where I can invent Ruby programs that most people don't expect to work, but they totally do. And I acquired it through building my Ruby auto formatter which led me to have this incredibly detailed knowledge of how the puzzle works.
CORALINE: What are you going to do with that? Are you going to use it for good or for evil or for awesome?
PENELOPE: [Laughs] Well, I guess good and awesome side is building a Ruby auto formatter. The evil side is that I can now win any trolling threads about the Ruby programming language anywhere on the internet.
JAMEY: I've literally already seen you do this at this knowledge. I was going to be like, "For evil, is posting about it on Twitter."
PENELOPE: Yeah. It is the most ridiculous review program you can come up or you're surprised this exists.
JAMEY: And then it's just like a bunch of letters and like semi-colons only.
PENELOPE: [Laughs] Yeah, basically.
JAMEY: I guess all words are just a bunch of letters and punctuation only.
PENELOPE: Yeah.
CORALINE: I can't wait for you to write a Ruby interpreter using white space. That would impress me.
PENELOPE: Sure, why not?
JAMEY: I want to congratulate you for the first person that I know of to come on the show and immediately describe their own super power as cursed.
[Laughter]
CORALINE: I think that is a first, yes.
PENELOPE: It is extremely cursed. Basically, I went into a cave and I got cursed. I came out the other side with this power that is both equally good and very, very evil.
REIN: Do you have like, so the Ruby grammar is written specified parse.y [Inaudible]. Do you have like, you know the matrix where they have like zeros and ones going down the screen? Do you have like snippets of parsed out why just like floating around in your head all the time?
PENELOPE: It's kind of like that but for Ruby files. I read a Ruby file and I can tell you exactly which parse nodes are being --
REIN: So, you see the matrix but for Ruby grammar.
PENELOPE: Yep.
REIN: So it is like the least useful version of [inaudible].
PENELOPE: Yeah, I think that's totally fair.
CORALINE: And then binding prize like think I need an exit.
[Laughter]
PENELOPE: Yeah, exactly.
REIN: I can see the matrix in this one very specific way that mostly annoys people.
PENELOPE: Yeah. I mean, the useful flip side of this is I can [inaudible] the existence of like pieces of Ruby grammars that I've never seen before and then write them and have them work the first time, which is how I discovered that any valid right hand side can go in a rescue clause in Ruby. So if you have rescue and then the FAT error you don't just have to put a local variable name there. You can literally put an entire clause definition in that position if you want to after the FAT error and then access a field on that clause definition. And that is completely legal in Ruby.
CORALINE: And I'm sure that would definitely make it through PR review.
PENELOPE: Yup. Absolutely. Why not?
CORALINE: Yeah.
REIN: It's like you showed me this on Twitter and I've been using Ruby for 15 years and I did not know that this was a thing because it's a stupid thing that shouldn't exist. And so, it's like syntax that is not actually like an object or an expression. It's like syntax that gets translated into [inaudible] calls like equals on whatever the thing is, which is sometimes a method call.
PENELOPE: Yeah. So if you want to go into the technical detail a little bit, I can absolutely expand on that.
REIN: I regretted that immediately.
[Laughter]
PENELOPE: Let me get my whiteboard out.
JAMEY: Oh, no!
REIN: I have made a huge tiny mistake.
PENELOPE: This is a Ruby interpreted design podcast now
REIN: I do have one more parse.y question for you though. And then I am happy to never talk about it again for the rest of my career. And that is when you were writing Rubyfmt, did you find any bugs in the Ruby grammar or its implementation?
PENELOPE: It depends what you mean by bug. Unfortunately, the simple answer is no. The complex answer is like 14 paragraphs of text about philosophically how Ruby parse trees should be constructed.
REIN: So, is this like the code is the spec kind of a deal? Like there is no spec for the Ruby grammar, so whatever parse.y does is the Ruby grammar?
PENELOPE: Yes, pretty much.
CORALINE: My understanding is that makes it very difficult for people working on things like JRuby or other implementations because there is no written standard for how Ruby is supposed to work. Is that still true?
PENELOPE: Yes. That's completely accurate. Philosophically, I guess the reason why I'm doing this, the way I am doing it is that there is no other implementation of the Ruby parser that you can just load into a program and execute. And so, JRuby has their own parser that is actually like a correct reimplantation of the Ruby parser but it's in Java so it's not like linkable. And then there's like the whitequark parser which is an attempt to reimplement the Ruby parser in C++. But that is not actually a fully accurate implementation of the Ruby parser, and they just gave up trying basically because they were like, "There is no spec for this. It's too hard to get it right." So, this is a good approximation for most people, but it's not truly accurate. So the reason why no one has done this before is that working with the Ruby grammar is just really, really difficult. And an essential side effect of this project is making that easier.
REIN: And you can't just rip parse.y out because it's like coupled to internals of JRuby.
PENELOPE: Yeah. It's funny you asked because I just looked into exactly what would you would need to do to be able to do this the other day. Basically when the CRuby interpreter boots, it does a bunch of global process level initialization things in C. So Ruby uses a stack based interpreter with its own object allocators, both memory-wise and in terms of the garbage collector. And all of that needs to be initialized before the Ruby parser can actually run. Because for example, when the Ruby parser parses a string, it allocates a Ruby string object. It doesn't just like slice out the string from memory. And so, you need like a partially booted Ruby virtual machine to be able to parse Ruby with Ruby's built in parser at all. And that's why it's so hard to extract out. And I'm at the very early phases all doing that work so that we can actually have, once and for all, a separate Ruby parser from the Ruby implementation.
REIN: Would CRuby ever switch to consume that parser?
PENELOPE: What do you mean?
REIN: If you would take parse.y out and you make it decoupled from the Ruby implementation, then could CRuby depend on your parser?
PENELOPE: Yes, it absolutely could. And some of the early conversations I've had with some of the Ruby folks suggest they might be open to accepting that. But this work is way too early to know whether or not that will happen.
REIN: Because that could help standardize the grammar across different implementations too.
PENELOPE: Yes, exactly. And in fact, for example in Sorbet, the Stripe type checker, Stripe actually decided that working with the Ruby parser in the virtual machine was going to take too much time for them. And so they actually forked the whitequark parser and basically did full re-implementation of the Ruby parser in C++. And the reason they basically gave me for that is they wanted an implementation, which they had full ownership of so that they could make changes they needed to frustrate Ruby in any case. And yeah, it's actually like a pretty interesting human story because the true underlying reason why the Ruby parser is like this is the backend, the one eight days Ruby interpreted Ruby Hoge just by walking the parse tree. And so a lot of the reasons it's like this are performance optimizations that Matz made in the really, really early days of Ruby in order to work. And so that was no external pressure for that to be a linkable version of the Ruby parser because the language just wasn't designed for this. And so, it's a really interesting side effect that because no one else was interested in the Ruby project in the early days of the development of the language, the parser has ended up being a thing that could only be used in the context of the Ruby virtual machine and not like on the outside. And so other teams are now building their own parsers for their own purposes.
REIN: So it's basically like if you want access to the Ruby AST, you have to first invent the universe, like at least 10 different teams have done that.
PENELOPE: Yes. I would say probably less than 10 but it's sort of close to that order of magnitude. I look at this situation, I'm like, "We're all trying to do the same thing here. Let's stop reinventing the wheel. I will just do the really, really hard work of pulling the parser out one time..."
REIN: [Inaudible] tribute.
PENELOPE: Yeah, just sort of attitude and only I am the kind of person where if you hand a really, really hard problem that seems tractable and no one is doing it because it's just -- the only reason people aren't doing it is because this is Hoge. But we know it's possible. We know we can do this with computers, so why not do it?
REIN: But it's also tedious.
PENELOPE: Maybe.
CORALINE: Penelope, the question that comes to mind for me is what problem are you trying to solve with this project?
PENELOPE: That's a great question. In order to talk about this, we need to talk a little bit about Rubocop, which is the other tool in the room. Rubocop is trying to sort of be the Swiss army knife, if you like, of working with Ruby source code. It's a formatter and a linter and it metrics and it does like bundler, Rails and [inaudible] cops, it does security. It does a whole bunch of stuff. And because of that, it doesn't do any one thing extremely well. As well as that, Rubocop has this sort of set of defaults, which I think based on my experience with all of the Rails teams I've ever met, no one is particularly happy with. But it also exposes every single thing that it does as a configuration option.
REIN: Every team I've ever worked with thinks some part of Rubocop's default are wrong, but they don't agree on which one.
PENELOPE: Right. And so, my observation has been that every single team comes up with that own Rubocop configuration and they sort of do it once and then they never ever look at it again. And this can have a couple of side effects. The first thing is Rubocop changes over time. And when it does, there may be new things you want to include or old things you want to get rid of. And the other thing is that your team and your organization changes over time too. And so the things you value about how your code works today might not be the things you value about how your code works in a year or two from now. And if you've got the sort of aging Rubocop configuration and people are taking it as sort of like a source of authority, that can become kind of problematic quickly.
REIN: And also there's a thing where it just like yells at you when you got stuff wrong.
PENELOPE: Yeah. And I guess it just sort of bounce off that. Rubocop can't actually automatically fix all of the things that it checks for, and frequently the error messages are not the most helpful about how to fix the problem that you encounter. I have had, since I started my project, about a number of more junior folks come up to me and be like, "I'm so excited for this because I keep arguing with my seniors to try and change the Rubocop file, and they weren't listening to me." And I'm like, "That sounds like you have a team dynamics problem. But if I can provide you a tool to solve your team dynamics problem, I will be more than happy to do that."
CORALINE: That is something that I have thought about quite a bit when you're establishing that initial Rubocop configuration. It's so impossible to do it by driving the consensus because there are just too many opinions in the room. So what happens is the loudest people or the people with the most political clout win the conversation and then that is encoded into this configuration file that, like you say, is long-lasting. And there is absolutely a power differential at play in how that gets created.
REIN: Let me put it this way. Do you know off-hand how many flags there are in Rubocop?
PENELOPE: I know it's strictly more than a hundred and strictly fewer than a thousand.
REIN: So what that means is there are more possible configurations of Rubocop than there are particles in the universe.
JAMEY: [Laughs]
PENELOPE: I guess, yes, that's certainly true. The thing is with this take, you could sort of reflect back onto me that if I'm making a Ruby auto formatter that has zero configuration flags, that I'm also silencing the least powerful people in the room. But what I'm actually doing is silencing everyone in the room. And the way I think about this is that designing specifically a code, ignoring all the other things that Rubocop can do, designing specifically a code formatter, the outputs Ruby source code that everyone can agree is consistently formatted and is something that could be called a correct formatting even if it's not everyone's favorite. Building that formatting is an extremely hard problem because there are so many little fiddly edge cases and when you move things on to new lines and how you do indentation and which constructs you use and that sort of thing. And I found very quickly that actually once you have built just like a functional auto formatter that can take source code in and spit source code out, there's at least as much work if not more in deciding how the source code actually looks. And I think that, to sort of come back to the point, Coraline, you were making, that the loudest voices like consensus is certainly the wrong way to make those kinds of very subtle and hard decisions because you actually just need to cook them over a long time in a single person's brain. It's not the kind of decision you can make quickly and rationally even through conversation. You need to look at lots of code, you need to format it, you need to see how it feels. You need to actually do a sort of more material exploration. And I think a lot of teams assume they can solve this in like a single two or three hour session and the problem is over, but actually everyone will end up hating it regardless.
CORALINE: The thing that I use Rubocop for, the only cases in which I use it are if I have a project that multiple people are going to be contributing to at skill levels that I cannot predict. And the main output, the two main things I want from it are a code base that is consistent, and I don't even care about the rules necessarily to a granular degree as long as there's consistency. But the other aspect of it, and this is like a human cost, is I don't want someone to be nitpicked in a PR review over syntax.
PENELOPE: Yeah. I completely agree with you. I think I sort of kind of have a hierarchy of nitpicking that I sort of came up with when I was thinking about this. The base level is you have no tool reviewing your code and formatting whatsoever. And so it's humans nitpicking humans. And what will happen there naturally just over time is the people at the top of the power dynamics will nitpick to death the people at the bottom of the power dynamics. I know a number of people. Betsy Haibel I think is a really good example of this of people who've just gone on [inaudible] talking about problem. And that's sort of the next level up. You kind of have the Rubocop universe where you have a configurable formatting tool where no one is nitpicking style preferences in a pull request because you either get a check mark or like a failure mark, but you can nitpick people's preferences at configuration time. So if someone wants to propose a change at configuration time, you get that same power dynamic except it's almost more ingrained because there's so much authority associated with the configuration of a tool like that. And what I've generally seen, and this comes more from my work on Go than it does my work on Ruby where they have gofmt and golint, which are literally not configurable. The language authors decide 'this is what code looks like, end of solution' is like there's no one you're interacting with on a daily basis where that power dynamic can apply. Now, the sort of up-level of that problem is then you versus language author and that can be really, really hard in those communities. I'm not going to sit here and pretend the people who are at the top of the go community have the best reputations for dealing with underrepresented minorities in tech. But I think assuming you have a good community like the Ruby community has, I think having those defaults be evaluated as a community basis by someone who is well known and well established is a really good way to solve this problem.
CORALINE: Are you collaborating with anyone on the decisions that you're making, Penelope?
PENELOPE: I very much intend to. Straight up right now, Rubyfmt is not in a place where it's ready for all but sort of like the simplest of decisions. An example of a style decision that has been made is the one I presented in my RubyConf talk, which is like to do with relative indentation to the parent of if statements and string embeds and case statements and stuff. It's kind of hard to explain that in an audio form. And to some degree, I'm working with Justin Searls to iron out a bunch of these cases. Sidebar, part of my motivation for doing this project when I started it was Justin sort of started work on his project which is called Standard. Standard is basically trying to be wrapped Rubocop and just decides the good configuration flags and then remove the ability to configure a Rubocop. I was like, "That's cool," but I really want a go format like tool where I can have it in my editor and it saves so fast that you can't even notice the formatting is happening, and let's at least make sure these things are compatible. And so, Standard -- I completely derailed my train of thought.
REIN: Standard is configuring Rubocop and making it so that there are no flags but you want it to be fast enough to work on the same.
PENELOPE:
Yeah, which you literally can't do with -- actually any tool that requires bundler to be present. Booting bundler takes somewhere in the order of 400 milliseconds and that's just too slow for interactivity.
REIN: So you're doing this in C.
PENELOPE: I am doing this somewhat in C and mostly in Rust. I have programmed plenty of C in my life enough to know that I hate debugging memory safety issues and the Rust programming language gives you native performance without having to debug memory safety issues.
REIN: If only there were some way to just not have to debug memory safety issues.
CORALINE: So Penelope, what are the major challenges your facing better not at the code level?
PENELOPE: That's a really interesting question. I think one of the biggest ones is that like the -- well, I guess this is really at the code level in a sense -- but the Ruby programming language has no implementation documentation. So I'm having to -- I don't know if this is what you were looking for. Actually I'll kind of say mostly I'm working on my own right now and just writing a lot of code and having no communication overhead and just building a thing is really nice. But yeah, I don't know that I'm actually facing any sort of like non-code level challenges with this project right now. Just time and motivation really. This is something I'm doing in my spare time not as part of my job.
CORALINE: So you haven't gotten any pushback or anything like that from the community?
PENELOPE: No. I would say if anything, the reception to this has been overwhelmingly positive. No one seems to think this is a bad idea, which is actually super encouraging. I guess one thing I did have, perhaps this wasn't a challenge so much as it was a misunderstanding is a lot of folks were like, "How do I install the gem?" And I'm like, "This is not a gem and it will never be a gem," which is like, it's the community standard way of delivering software. But the reason I'm pushing against that is it's kind of like a lot of people were expecting this to be distributed as a gem, which is like the Ruby standard way of distributing packages and it's not a gem. And the reason for that is like a whole host of implementation reasons the aren't worth getting into. But it was really interesting to me to be building this Ruby thing and it being a bit nonstandard, and people having such this overwhelming expectation of how it would be distributed. And me being like, "Nope, I have my reasons. Please believe me these are good reasons to not do it this way. It will be fine, I promise." And actually it's really interesting because the Sorbet team ran into the exact same problem. All of the code for Sorbet, it's a native C++ program that they could just compile to an executable on both Mac and Linux. And they also created a gem, even though there's no reason, there's literally no technical reason why that needs to be the case. They said that they did that just so that people would have it in the normal workflows and they could instruct people to upgrade and do a bunch of those distribution things in a way that they're used to doing. And it was very interesting to me to be like, "This is not the same as the things you are used to receiving in the Ruby community," and so I'm distributing it in a different way. And there was quite a lot of confusion around that.
CORALINE: Penelope, how will that actually work when you are finished with Rubyfmt then other people will have to take that and adapt it somehow to a variety of editors using a plugin architecture? How is that exactly going to work?
PENELOPE: There will be editor plugins for all the major editors. Actually, it's really funny. People started building editor plugins long before it was ready for any kind of daily usage. So I will say, and this sort of comes back to the response that I've gotten subject is like I was overwhelmed by how positive the response was. There are I think vscode and Atom plugins for Rubyfmt already that I did not write. I don't use vscode or Atom. I also have no idea how good those plugins are. And so, I was just like, "Wow, okay. This crashes on a certain percentage of Ruby files today, but good for you for sticking with it." I mean, the core of it will just be a binary, like a Linux or Mac OSX binary that you download. There will be a website where you can pull updates. I will probably ship an update flag in the binary as well just to download it or compile from code. But almost none, in fact probably by the time I am done, none of the code in Rubyfmt will be Ruby, and so it doesn't make sense to try and distribute it as a Ruby gem.
CORALINE: Yeah, that makes sense.
REIN: So, like OS package managers?
PENELOPE: Yeah, Brew or cURL piped to pseudo bash. You know, that old Chestnut.
CORALINE: What could possibly go wrong with that? So, Jamey, you had a question?
JAMEY: I do. I think it's really interesting that Coraline asked what are the non code challenges and you answered like. "I haven't really had non code challenges other than finding the time to work on this personal project," or whatever which I think is one of the biggest non code challenges that any of us face ever. And so, I want to ask about that. I want to ask about like how have you carved out time for this project and how do you motivate yourself to work on something like this. I mean, you've done lots of projects. How do you motivate yourself to work on these things in your spare time outside of all of the other things that you're doing?
PENELOPE: That's a great question. It's a little hard for me to pinpoint exactly what it is. Part of this is like Rubyfmt is genuinely, I think, the hardest thing I have ever done with a computer by some distance, and I'm loving bathing in the technical complexity. So, some of it is that for sure. I think having a thing that is truly mine to just sort of pull on has been great. When I was working on RSpec, I was one of several maintainers, and so I felt some ownership but it's not the same thing as having complete and total ownership, which has been huge for me frankly. It really depends. If you look at the GitHub [inaudible], there will be entire months where I don't do anything and it's fine. And then I'll have like months where I write literally thousands of lines of Rust code. And so my personal motivation definitely swings. I don't want to get super far into it, but I also sort of medically and socially transitioned over the last nine months and that has been part of the development work. There have been days where I've just been like, "Hormone's too overwhelming, I'm going to sleep now," instead of working on Rubyfmt. And so, it's really variable to be honest with you and a lot of it just depends on my mood and whether or not I feel like attacking the problem today, but it's a lot of fun.
REIN: Penelope, you were saying that you really haven't gotten pushback on your plans for Rubyfmt. I was wondering if you could maybe try to elaborate on what you attribute that to?
CORALINE: Penelope is a superstar and everyone wants to do what she says.
REIN: Yes, and what else?
[Laughter]
PENELOPE: If I were to guess, it feels so -- I mean, I can't tell what's in other people's heads. But I can tell you why I wanted to do this. I've been thinking about a tool of this form for close to five years now. I've been thinking about building something like this for actually quite some time. And the reason I never really got started is I didn't feel any sort of pressure or need because the Ruby community sort of had consensus on Rubocop and there was no other tooling being developed in this space. Literally, no one was doing anything. And then last year at RubyConf, Justin came along and was like, "I'm going to build Standard and here's why I think it's a good idea." And I was like, "Okay." His was somewhat personal story. I was like, "I can't live with the idea that we are all using Rubocop as like the final formatting thing." Because the way Rubocop interacts with the Ruby parser just can't do the level of detail I want a formatter to be able to do. This is also like my deep and personal need for perfection. I've been thinking about what a tool of this form would look like for, as I said, about five years now. It was really inspired by when I saw gofmt working. So when I first worked in the go programming language, I'm like -- if you've not ever worked in the go programming language, basically the out of the box editor tool chains all set up auto format on save and you save on your file just snaps into place. Like the code just becomes correct and it's this absolutely beautiful feeling. It's kind of like little dopamine hits every time you save a file. And I know folks in the go programming language who have sort of said like, "For me, if I'm coming to another programming language now, having a tool like this is absolutely table stakes." Because I don't know how folks are laying out their source code in those languages. And if there's a tool that does it for me, that's so much the better. One of my actually biggest people who's been cheering me on is a friend of mine. He's like a very experienced go developer who occasionally has to touch Ruby. And he's like, "I always get style nitpicks on my pull requests, please make them go away." And I was like, "I will absolutely do that." And so, I think if I were to sort of point out why the response has been so positive, it's that there actually isn't another tool in this class for Ruby today. Rubocop is close, but it's not doing the same. Or certainly, it's not doing what I am promising my tool will do when it's done. An open question is whether I can do that. I think I can. But the other thing I should say is it's not certain. And so partially, I think this is also a hype cycle and I've clearly just hit on a thing people really, really want that I have really, really wanted for a long time and now, I'm actually dragging myself over the hot coals of working with the Ruby parser to do it.
CORALINE: I think that work is greatly appreciated, Penelope. I know I'm personally very excited about this because I did work in go about a year ago and go is the language that makes me happy. I know it's really good at some stuff, but I just don't enjoy writing it. But gofmt was magic and it made learning the language so much easier.
PENELOPE: Yeah. And that's part of it as well. One of the things I observed with my working in go actually, and you've sort of hit the nail on the head is learning the language, is like the first time I would write some code, I would lay it out and fire a pull request up and [inaudible] fail CI for formatting reasons or linting reasons. Because they also have a standard linter built in. And I would slowly learn better how to write go code to the point where I was almost always doing what the formatter or the linter would want before I even ran. That as a teaching mechanism I think is really interesting and we don't have tooling like that in Ruby. And I'm sort of taking on the grand challenge of building, not just the formatting tool, I want to ideate and build that teaching tool in the future at some point as well.
REIN: I have maybe like a somewhat heretical view on this, which is that the fact that we're even writing source code in text files as sequences of characters that then need to be formatted is a weird and unfortunate historical accident.
PENELOPE: Yes.
REIN: That's it. That's the whole take.
PENELOPE: [Chuckles] Okay. Let me ask you a question. What would you have us do instead?
REIN: [Sighs] I want an ergonomic way to work directly with ASTs.
PENELOPE: Cool. Would you preserve the existing programming languages or --
REIN: No, they're all forced up against the wall. [Laughs]
PENELOPE: Okay. So, we would have to have a new programming language explicitly designed with an ergonomic AST.
REIN: I mean, it's not going to happen, but yes, that's the thing that would never happen.
PENELOPE: Cool. So Rein just reinvented Lisp.
[Laughter]
REIN: Funny thing about Lisp is that it was originally supposed to be the end language that they were going to write an ergonomic thing on top of.
PENELOPE: Yeah. I would put to you that if Ruby developers were forced to hand edit Ruby ASTs instead of Ruby source code files, there is no way to make that more ergonomic.
REIN: No, that would be terrible. But I think something can be done that's better. I just don't know what.
PENELOPE: You can theorize programming languages I think where that would be the case, but taking existing ones and trying to make that happen would be extremely painful. For example, the way C and C++ and Ruby and a bunch of other programming languages parse like, list like, or hash like structures is with tail recursion. And then you have to teach a whole generation of programmers how to do tail recursion.
REIN: I told you that was heretical.
JAMEY: [Inaudible] label your take. Thank you.
PENELOPE: I like this take. It's a take I have had in the past as well. I just think that the practical way we get that is unfathomably difficult and starts in CS research in universities and takes 50 years to leave that space.
REIN: There is a little bit of work that's being done with editors that actually sort of guide the code as you write it rather than waiting until you write a bunch of stuff and then save it. So, we'll put the cursor where it needs to be for the thing you're currently doing.
PENELOPE: Yup.
CORALINE: I believe I would hate that a lot. Penelope, what else do you want to talk about?
PENELOPE: One thing I think is potentially interesting. I don't have answers here. I have questions more than anything. So at the Ruby Central directors' retreat, right before RubyConf, we were talking a little bit about how we were lamenting the death of regional Ruby conferences. It really feels like they've all disappeared. I don't know if this is also happening in other programming languages that have been around for as long as we have or what's going on there. We had some sort of theories. But my question is, is having regional Ruby conferences a good thing in the first place and if it is, what can we do to bring them back?
CORALINE: I can say for a lot of people, regional Ruby conferences can launch their speaking career. There's only what, 35 slots for RubyConf.
PENELOPE: Sixty, including keynotes.
CORALINE: Yeah, a very limited number of slots available. And that conference is hard to get into. It took me five years to get accepted at RubyConf and I speak all of the time. So, regional conferences are how I got my start, and I think they're valuable for other people. I do miss them. We used to have so many five years ago and now there's just a handful. And I think some people interpret that as a reflection of the language itself. But I wonder, I guess like you were saying, is it an evolution of the community?
JAMEY: I do a lot of regional conferences, not a lot, but I tend to do regional conferences including ones that aren't like where I'm from because I'm jay speaking. But like DevOps days I think is a great example of a community that's still doing tons of regional conferences. And I think that what's really interesting for me going to them, particularly going to them in other places. And being an out-of-town speaker among attendees that are mainly local is kind of an interesting experience for me. But it gets me thinking of like -- I agree with what Coraline said about speakers getting their start, but I also think that it's like an accessibility thing for folks that can't afford to go to a conferences out of town. And so, like a conference comes to your town and you go, and I've noticed this in other places, but like Buffalo just had our first DevOps days. And I had been doing a couple of DevOps days conferences and it was kind of neat talking to like, I didn't know a ton of the people there because Buffalo doesn't have like a super tight knit tech community. But it was mostly local people and it was mostly local sponsors. And I was surprised by that and it was a cool experience. And so, I think that that is really valuable and that is something that I think is missing if we're congregating at these bigger events where mostly everyone is traveling.
PENELOPE: Yeah. So I really only know two, I have two like one person anecdotes about this. So, one is that a local conference organizer who's been doing it for a number of years told me they were struggling to get enough speakers to fill that conference and another is that organizers just burned out. And there wasn't a generation of organizers to replace them. I guess my working theory is that we are potentially responsible for creating a whole new generation of local conference organizers and making it. And here when I say we, I mean Ruby Central as the sort of position as the steward of the community, creating a new generation of conference organizers who can run those conferences. Because to me that seems to be the thing that dried out.
REIN: A thing that I've heard from a few people who've run or ran these regional conferences is that they weren't really trying to make money but it was hard to even breakeven.
PENELOPE: Yeah.
JAMEY: Yeah. I was also going to bring up money. I've been an organizer on a regional conference and I'm not one of the main organizers, so I don't deal really heavy in the sponsorship stuff. But I know that it's a worry and I know that we didn't have our conference this year because we didn't have the money to have it. And we kind of went, "Well, this is still important to us, so we're going to regroup and start earlier and start looking for sponsors for 2020." But in an ideal world, we wouldn't have had to like not have our conference in 2019.
PENELOPE: I think the thing that I have heard that's often really scary is many conferences don't breakeven, they make a small loss, but they only get really close to it right before the conference happens if they do breakeven or like make a small profit. And so right up to the deadline, you're looking at a huge financial shortfall. And for individuals, that can be extremely scary.
CORALINE: Is that something Ruby central could [inaudible].
PENELOPE: I don't know. We'd certainly have to review that.
REIN: So it sounds like the things that are needed for regional conferences to be sustainable are sponsorships so that they can not feel super stressed out about a financial disaster because they're trying to bring people together to talk about a programming language. Is that the main, or what are the components?
JAMEY: I think another thing that got mentioned was lack of speakers. And I think that as a speaker and someone who like, I actually really quite like to do small regional conferences in other places, but I can't afford to travel for conferences that can't pay a travel stipend to me. And I haven't been at a company that can help me out. And so, it's not that I don't want to do it and it's not that I'd begrudge them necessarily for like -- being in a situation that I know I've been in as an organizer where we don't have money, but it's just kind of unfortunate. I think if more money was there, that could alleviate some of these problems, even more of these problems than is immediately apparent potentially.
PENELOPE: Yeah. You were sort of arriving at this really interesting point, which is assume that was like financial oxygen such that any organizer knew that would be no financial burden on them whatsoever, would that change the dynamic?
JAMEY: I think it would.
PENELOPE: Yeah. And I think the answer is probably yes because I know when there's at least one regional conference that I'm thinking of, that I was about to say the name off but I'm not going to, that spun down essentially because it stopped being financially viable for them to run the conference. It's really interesting.
REIN: Do you think that the DevOps days regional conferences are sustained by having a lot of developer relations folks with people whose job it is to go to conferences speaking at them?
PENELOPE: I don't know, I wouldn't be surprised -- because it does a lot of money sloshing around in that ecosystem. It wouldn't surprise me at all if like the top -- you could probably get five to 10 reliable speakers just because that company will pay for any amount of travel. All the big players in DevOps have the travel budget and I guess that's less true and there are fewer dedicated Ruby developer evangelism teams with budget these days.
JAMEY: As someone who speaks at DevOps days conferences, again, I pretty much go to ones where they can pay me a travel stipend because I don't have that. But I think it's possible that the money that's going towards my travel could be being kind of bankrolled by other speakers who aren't asking for that financial assistance. And I don't know about that, if that makes sense.
REIN: So how do we get avocados to come to Ruby conferences?
JAMEY: I feel like what I would prefer over that -- I mean, I am a developer advocate, so I'm not like making fun of the avocado thing. But I've talked a little bit about organizing and I haven't said what conference and I will. I'm an organizer for Catskills Conf. And what we've tried to do, like particularly it takes place in the Hudson Valley. I'm not located in the Hudson Valley, but most of the organizers are. And they push really hard for local people to be speakers. We don't have a lot of speakers coming in from out of town. I'm coming in from Buffalo, which is a 5-hour drive. And I'm usually one of the farthest away people coming, including the speakers. And I think that it's been a good experience overall. I mean, obviously as I said, we're still struggling with sponsorships and stuff because I think that's just a thing that goes around. But it's been a great event and it's been a good perspective to have people from that community talking about tech. But like talking about tech in terms of that community has been, I think, rewarding for that group of people, if that makes sense. And I would love to get more people involved in speaking rather than making a point of trying to get the same people over and over in the way that you're describing.
PENELOPE: And I think at the very height of the Ruby conference regional days, that was a little bit more of that. I know I used to be able to do 10 to 15 new conferences in a year, without even having to try all that hard. And that was just because they were like, "Hell yeah, come give the exact same talk in 10 different cities." And I was like, "Cool." But there were loads of people doing that. There were people who I went to four or five or six conferences within the same year and also gave the same talk four or five or six times. And so, I do still wonder if it was like lack of growth of local talent and external talent sort of pushing that out a little bit.
REIN: I think it would be unfortunate if the only way to sustain these local conferences is also pushing out local speakers and things like that.
PENELOPE: Yeah.
REIN: Cool, super upbeat way to end the podcast.
[Laughter]
REIN: I really missed -- I love how diverse they were. The cities are all different obviously, but they were not all in big convention centers. A lot of the venues are cool and unique and just a whole bunch of people talking about stuff that you wouldn't get to see at RubyConf like Coraline was saying. So, I missed them.
PENELOPE: It was funny, I was just talking to Ernie Miller yesterday and he was lamenting that he feels like his favorite talk to give is not one he could give at a big conference because it's too risky. And I was like, "That's such an interesting way of putting it." And then he told me a little bit about the idea and I was like, "You're right, I would stop you from giving that talk at RubyConf because it would almost certainly go wrong in horrible, horrible ways," and there is no backup strategy for that. So yes, I agree with it. You got riskier, more interesting talks, less hermetic.
CORALINE: Less of the same people speaking over and over and over again.
PENELOPE: Yup.
CORALINE: I love Aaron Patterson, but he speaks at every conference.
REIN: [Inaudible] Being very funny though.
CORALINE: That's true. But there is a core population of speakers that speak at every major conference and I too miss the quirky, the local flavor, the up and coming people. I knew for me it was a career changer and I hate the fact that that is closed off to so many people now.
PENELOPE: And I think that career changer point is the one that's perhaps most important is I don't think I would be where I am today in my career without access to -- and I guess to bring it back, speaking at 10 to 15 conferences per year, that put my name in the ears of literally thousands of people that I would otherwise never have met.
REIN: Yeah. They changed a lot of people's lives, myself included.
PENELOPE: Oof.
REIN: Cool. It's even worse now.
PENELOPE: I'm sorry to have created a big sad [inaudible].
[Laughter]
JAMEY: I'll say something kind of a little bit less sad, which is that conferences and doing speaking has changed my life too, and I kind of am not old enough in the industry to remember some of the things that you are all talking about. By the time I started doing conferences, a lot of this was already on the decline and it's kind of sad to listen to you all talk about something that I kind of missed out on. But I think in other ways, I don't know, I'm having trouble with what I want to say because I'm not the expert, I guess. But I've had a pretty good experience at conferences and my experience has been probably getting better since I started a few years ago, in that more conferences seem like they care about new speakers and diverse lineups. And I hear some kind of this, "Oh, the good old days," with how conferences used to be and these smaller events and this. This isn't the first time I've heard that. But I've also heard a lot of horror stories about how things used to be and I think that the conference culture that I'm in now, I like it.
CORALINE: It definitely has changed. I mean, it was only five short years ago that we had raging debates about whether conferences should have codes of conduct.
PENELOPE: Yeah, that's true.
CORALINE: That was her real fight.
PENELOPE: Yeah. And like, the Ruby Central conferences, we provision gender neutral bathrooms and a childcare facility. It's now completely consistently. That certainly wasn't the case five years ago either.
REIN: Yeah. To build on this and maybe bring it back to a happier place, maybe if not like there was this prelapsarian time when the Ruby conferences were very good and that's all lost now. Maybe it's just that the industry is changing and we don't have those conferences anymore, but we do have other regional conferences in different parts of the industry, and there may be even better than what we had before.
CORALINE: Yeah. I mentioned something about that because I found that I have shifted from local Ruby conferences or regional Ruby conferences to regional Polyglot conferences. And I actually wonder if that might be better because for a long time, the Ruby world was pretty insular.
PENELOPE: I could see that.
REIN: For myself, I, like for a decade, considered myself a Rubyist. That was like the primary thing for me. And only in the last five years or so, I really started with say, "I'm an engineer who knows Ruby."
PENELOPE: Yeah.
CORALINE: Well, Penelope, it's been an absolute delight to talk to you. I'm so proud you came on the show.
PENELOPE: I'm glad. Thank you.
JAMEY: Penelope, I'm so glad that you came on the show. It's so wonderful to have you on and for us to all be able to chat and it was delightful. So, thank you so much and thank you to everyone for listening. If you want to continue with some of these conversations with us, you can always back us on Patreon and that will get you access to our Slack community. Everyone who's ever been on the show has access to this community and we have great conversations in there and we'd love to have you. Thank you.
