Episode 062: 2019 Flyover Camp With Tess Flynn
Our frequent DrupalCamp attender and speaker, DevOps Tess Flynn, returns to the podcast to recap her recent experience at Flyover Camp, a brand new Drupal camp in Kansas City, Missouri.
Host: Ivan Stegic
Guest: Tess Flynn, DevOps Engineer at TEN7
- We in the midwest totally own the “flyover” jokes!
- The continuing diversity in camp talks (business, self-care, human focus tracks)
- Tess reviews both her talks (Return of the Clustering: Kubernetes for Drupal, and Health Check Your Site)
- How you should stretch your mind to prepare for all the rapid-fire information you get in the Kubernetes talk!
- Location, location, location is as important for conference talks as it is for real estate
- Listen to Ivan and Tess geek out over the Raspberry Pi session
- "Everything I Know About Kubernetes I Learned from a Cluster of Raspberry Pis" (Jeff Geerling)
- "45 Modules and 45 Minutes" (John Rearick)
- "CloudWatch-ing" (Rhia Dixon)
- "Return of the Clustering Kubernetes for Drupal" (Tess Flynn)
- "Dr. Upal Is In - Healthcheck your Site” (Tess Flynn)
- Twin Cities Drupal Camp
IVAN STEGIC: Hey everyone you're listening to the TEN7 Podcast, where we get together every fortnight and sometimes more often, to talk about technology, business, and the humans in it. I'm your host Ivan Stegic. Let's talk about Drupal Flyover Camp 2019, that happened from Friday the 31st of May to Sunday, the 2nd of June, in Kansas City. Joining me to give her thoughts is socketwench. That's wench, not wrench. Welcome back to the podcast.
TESS FLYNN: Hello.
IVAN: Now, did I say it the right way, because I know you always have a specific way of saying it when you give your intro to socketwench.
TESS: Well, that’s pretty close. That works.
IVAN: Close? Okay. Good. So, you were at a Flyover Camp. What's in a name? I just love how Flyover Camp were poking fun at themselves in Kansas. I mean, we're pretty much in flyover land here in Minneapolis too, so I totally get it.
TESS: [laughing] So let's first frame what that is because if we're having international listeners, they might not get what the reference is.
IVAN: Good idea.
TESS: So, the thing that goes with it is, if you're from the Midwest you're considered in flyover country. And the reason why is because the joke goes, that there is nothing in the United States that's of interest unless if you're on either coast, which is actually completely untrue. However, that is what a lot of people tend to think of it. So as a result, if you're in the Midwest you kind of go, Well, you know, what we're going to own that turf.
TESS: We're going to go and names things after it and take that world.
IVAN: I love it. I love that they did that. Drupal Flyover Camp in Kansas City, Missouri. And so, this is a brand-new camp, right? This is the first time they've ever done this camp. How great is that? We have a new camp on the schedule.
TESS: Yeah, I was surprised that it was new because they hit everything running. It felt like this was a well-oiled machine for a camp.
IVAN: That's wonderful. It's wonderful to have that on the calendar again. So, well-oiled machine. Did you recognize any of the organizers? Maybe these people have done it before.
TESS: I think that I recognized a few people from…oh, what is their name…VML and YL. What are they called now, because they merged with somebody?
IVAN: I don't know.
TESS: VML Y&R. Wow, that is a mouthful.
TESS: Victor, Mike, Lima, Yankee and Romeo.
IVAN: Okay. What are they, a global marketing agency that needs a new name? [laughing]
TESS: [laughing] That is their new name. [laughing]
IVAN: [laughing] Okay. I don't even know how to say it.
TESS: They used to be two different companies that got merged, and this is the resulting name.
IVAN: Oh, it's on their BOF page. If you're looking for something about VML you can still visit the VML website. If you're looking for something about Y&R, don’t sweat, you can still visit the Y&R. So, it's basically like you said, a concatenation of their former names. Maybe it's just temporary. Ok. A little bit of a tangent. [laughing] So, some sort of experience in Flyover Camp organization. Sounds like you said they were a well-oiled machine. It was a three-day camp?
TESS: I believe so. There was a day of trainings which I did not attend, and then two days of sessions, which actually has been bucking the trend lately.
IVAN: Yeah. And also, from what I can tell there were contributions as well on Sunday so, maybe it was a four-day camp, if there were trainings as well.
TESS: Might’ve been.
IVAN: Yeah. So, you were there Friday and Saturday. It looked like they had numerous tracks. So, I thought, usually these camps have five tracks and then you have five rooms and people go to the room for the track that they're interested in. This felt like it had a dozen tracks, but three rooms and it sort of was interspersed track sessions and BOFs as well amongst these three rooms. Is that what it was like? I mean I’m only gauging from the website.
TESS: So, you know the thing with the tracks is that a lot of the time it depends on how promoted they are as their own top-level entity in the data, as it were. And some camps do a very good job of this, that they have this track, this track and this track. I think DrupalCon recently reorganized so that there's only particular tracks that they directly advertise to different audiences, like a business audience, a frontend audience, something like that. Some camps have a lot of tracks and they're not particularly consistently organized, or if they are, it doesn't feel like that when you're attending because you don't tend to notice it, and Flyover Camp seemed to fall into this latter category. That's not bad but it's just a thing.
IVAN: Yeah. And I love that the tracks were so diverse as well, right? There was security, QA, site building, the usual frontend/backend stuff and there was a self-care track as well. I mean, more of that please. More mental health stuff, more business stuff, more human focus sessions. I love it.
TESS: Mm hmm.
IVAN: I love it. I think that's awesome. And, it looked like there were about 30 sessions, so similar to Drupaldelphia and those 30 sessions were spread across two days as opposed to one day at Drupaldelphia.
TESS: Yeah and it seemed to attract a lot of people from the area. I mean I was there from Minnesota and I saw people that usually I see at DrupalCorn there as well. So, it attracted a lot of people from the Midwest.
IVAN: That's wonderful. And there were BOFs as well, and it kind of looked like they were spread out across the two days as well.
TESS: I think there were, but I was so focused on other stuff that I completely missed it.
IVAN: Yeah. This was a heck of a camp for you. I mean it wasn't one session it was two sessions.
TESS: Oh man, it was a double feature. That was hard.
IVAN: I'm sure you absolutely shone on that and I'm sure you did really well. So, let's talk about those two sessions. So, your first session on Friday was the famous cloaked talk, Kubernetes called Return of the Clustering, right? The third part of the trilogy. So that was Friday. And then Saturday you gave a talk essentially about the Healthcheck module, right? What can you do to keep tabs on the health of your Drupal site?
TESS: Well, it was also about site auditing as well, in general.
IVAN: That’s right, and site auditing. So, I guess the critical question here is, did you wear a costume for both talks?
TESS: So, here's the problem with that. I don't have a car. And in order to actually get the costume for that one I would have probably had to rent a car to go to a local thrift store chain called Ax Man surplus and see if I could find like a stethoscope or whatever that little satellite dish head gear thing that they wear, I forget what it's called, and see if I could shove one of those into my luggage. But I didn't have the time to do that. Every weekend that I've had lately has just been completely booked up.
IVAN: Well maybe we'll have to work on that if you get asked to do that talk again and we'll figure out another costume for you.
TESS: Well, rumor has it that's going to happen. [laughing]
IVAN: [laughing] So, comparatively, how were the two sessions attended? Was there a drop of people on Saturday compared to Friday or was it comparable?
TESS: It was actually the other way around. I think a lot of people find the Kubernetes talk is fun, but it can be very intimidating because it seems like, “oh, that's a very devopsy, very technical talk and it's going to be way over my head.” And I was able to attract some people to come to it, especially by making a fool out of myself, by dressing up like a Jedi and standing outside of my door waving a lightsaber to have people come and join the session. But, it was a smaller room and it was still well attended, but the site audit talk actually had a lot more people in it, mostly because it was also in the main auditorium, so a lot of people who were just there were also just there, but there was a lot of people paying attention to it as well, because it tends to be a really fun, engaging talk and it tends to appeal to a much broader audience than the Kubernetes talk, which tends to be more infrastructury devopsy people. Even though I try to make that as broadly appealing as I can.
IVAN: So, location, location, location. Right. You had a wonderful location in the auditorium for that talk.
TESS: Yeah. The only downside is that when you're in an auditorium you're usually on a pedestal or a dais or something like that, and the problem is that it sounds like I'm a T-Rex walking around on stage, because the thing is hollow so the microphone just picks up everything, and I don't tend to stay still when I give a talk, I tend to gesticulate and walk around and do lots of weird things.
IVAN: Jump around I believe you do as well. [laughing]
TESS: [laughing] Yeah. Well, I think the site audit talk, I also fall to my knees at one point, dramatically. [laughing]
IVAN: [laughing] It's a good talk.
TESS: I still remember skinning my knee at DrupalCorn.
IVAN: Well, it’s a good talk. I think it gets valuable as you do that. It certainly reminds people how important it is. Right? So, what do you think the biggest question was that people had from that health check talk, from that audit talk?
TESS: You know, I didn't get many questions. I was actually thinking about this a few days ago. I tend not to get that many questions directly after a talk because usually my talks last the entire amount of time, and afterwards, I have to rush out the door for the next person to start setting up their talk. And usually I don't get many questions, and I do try to anticipate a lot of the potential questions as well within the contents of the talk. So sometimes people will come by and ask me questions later, but that hasn't happened lately.
IVAN: It's a similar sort of thing for both talks then.
TESS: Yeah. I did have a nice conversation with someone, I think they're from the U of Kansas. I forget. I remember their face. I know that they go to DrupalCorn regularly too, but they were telling me about Kubernetes operators and all of that nifty technical stuff and that was a really interesting conversation to have, but it really wasn't a question.
IVAN: Well that actually leads me into my next question. Usually you're the one educating people about whatever you're talking about. What do you think your biggest takeaway from a session was in Flyover camp? What did you learn from each of them?
TESS: Ok. Geez, I’m trying to remember all the sessions I went to because Twin Cities Drupal was last weekend, and now I'm trying to remember any of the sessions I went to.
IVAN: Oh, I think maybe you misunderstood. What I meant was, what did you learn in your talk from the audience?
TESS: Oh. So, one thing that definitely occurred to me is that, when it comes to the Kubernetes talk is, just how much technical knowledge you need, all technical terms you need to pick up very, very quickly to get anywhere with understanding Kubernetes without feeling like you're “drowning,” in technical terms, all of a sudden. And I certainly had that experience myself just trying to learn Kubernetes in the first place and that is after having a very strong background in how containers work and how Docker works and some of the top terminologies I picked up from running production workloads in Dockers form, and I realized that after that talk, like, wow, in 45 minutes I take you from, you kind of, sort of know what Docker is, and you might have heard of Ansible, but you don't know too much about it, to, here’s how you run a Drupal site in production on Kubernetes using a simple effective formula. And that kind of struck me as Wow, not many people are doing that because, wow that can be really complicated.
IVAN: Yeah, it's the bleeding edge of it isn't it?
TESS: It's not just the bleeding edge, it’s just that the underlying design that I went for strives for minimalism and simplicity, and a lot of people find that appealing because it reduces the number of working parts that you have to know. A good example would be memcached. The way that it's presented in the talk is as a stateful set and that works. A lot of people will say, "What you should do is run it as another object called a daemon set." But in order to introduce a daemon set, I'd have to introduce a completely new object type that only works for that, and afterwards it's like, "Is that really necessary to talk about it?” “How often do you add or remove notes?” If you are already thinking about adding and removing notes, you're probably going to look up this stuff for you. So, I don't need to actually tell you about this in this talk. [laughing]
IVAN: Yeah, I love that you're able to educate people in one session even at a very high level. To go from, kind of knowing to, being interested in the technology and in what we're doing and in being interested in continuing to find out more. And, maybe that's a good reason to do a separate podcast just on the talk you gave and the contents of the talk and why are we doing that? Why is TEN7 investing as much as we are in Kubernetes and in Docker and in Drupal, and, you know, sending you to all of these camps, and then putting all of this work into the open source domain? Like, maybe there's enough there to talk about. I mean, just from my perspective, we want to be independent, and using a hosting solution that is supported in the open source that is vendor agnostic. And, if we're doing it for ourselves, there's no reason why we couldn't put it out there and have others learn and leverage from it as well. So, we should probably talk about this a little more in a separate podcast.
TESS: That's not a bad idea at all.
IVAN: I love it. Okay. We'll do that. We'll ask Jonathan to make that happen for us. Okay, so, a little more about Kubernetes. I was looking through the schedule of talks and as you, Tess, know, Raspberry Pis are really near and dear to my heart. I've used them for many different things at home, most recently as an ad blocker for the whole network, but I saw that Jeff Geerling was at Flyover Camp, and he had a talk about the cluster of RPis, or the Raspberry Pis, that he's been building since 2012-2013, something like that, and how it taught him everything he knows about Kubernetes. Did you catch that talk by any small chance?
TESS: I actually did go to that talk.
IVAN: You're kidding?
TESS: Because I was like, Oh that sounds really fun and I'd like to see what he does. Is he going to use straight K8s or is he going to use that K3 that I heard about? And, what was funny to me is that I remember watching a talk that he gave, not about Kubernetes, but about Ansible. Way, way, way back in the day at MidCamp with a very similar block of a Raspberry Pi cluster in a box. And I really wanted to see what he was going to do with this. So, sure, I went to it.
IVAN: And was it everything you wished it could be? I mean, I looked at the slides and there was a shout out to socketwench in one of the slides.
TESS: Yeah, I was like thankful. I was in the front row and no one could see how I was blushing [laughing] the entire time. Like, Oh, stop talking about me please. This is your talk. [laughing]
IVAN: [laughing] That’s great. I mean, you guys are related and connected by Kubernetes, so, how wonderful that that would be the case. So, can you give me a quick synopsis of the talk? What was the nugget that you took out of it?
TESS: So, what was interesting is that Jeff has built a small Kubernetes cluster using a standard distribution Kubernetes to run on, I think it's four or five Raspberry Pis.
IVAN: I think it’s four.
TESS: I think it's four now. Four Raspberry Pis with a single ethernet switch with power over ethernet so that it reduces the amount of additional circuitry and cables he has to carry around to power them altogether.
IVAN: Hold up. Hold up. He's actually powering the RPis now through power over ethernet? That's amazing. Of course, you could.
TESS: You can get an adapter board for that.
IVAN: That's awesome.
TESS: It's not really complicated.
IVAN: That’s so great. I'm sorry. I totally interrupted you there. What was the nugget?
TESS: [laughing] Well he didn't even mention the power over ethernet except for one thing, but I was looking at the screenshots like, Oh, you’re using power over ethernet now. Nice. [laughing]
IVAN: Nice. That’s nice.
TESS: So, a lot of the talk was about how he was running his own personal site, using a Raspberry Pi cluster out of his home network. And, I used to run my own single node server out of a home network way back, many, many years ago. And there's a number of challenges that come with that out of the box. You tend not to get static IPs from most ISPs. They'll get a Dynamic IP, some of them don't like that you have a significant amount of outbound traffic or incoming traffic that's coming from the net and they may block you for that reason, if you're on a particular service tier. Some ISPs are better at that than others, it really depends. But running his own site on a Kubernetes cluster on Raspberry Pis it's like, it reminds me of this meme that I saw passed around Kubernetes Twitter a while ago, which is, the subtext is, I deployed my blog on Kubernetes and it's this big semitrailer and it has a toy truck trailer box in the middle of it, completely dwarfed by the full size trailer. [laughing] That’s kind of like, Yeah that's pretty accurate.
IVAN: Well, I mean if you ever get reddited or slashdotted, I guess maybe it'll survive?
TESS: [laughing] Kind of. There’s a degree of front side caching I think that he also used. This kind of a project always comes across to me as not a serious, You should use this instead of traditional hosting and more like, I wanted to see if I could do that and it would be fun and it's something to do and it's something that lets me learn by doing. And that's you know, a worthy pursuit in its own right.
IVAN: But if you look at the other side of that coin, you're hosting your own website, you own the hardware, you own the software, you own your site, you can see it, you're not putting the risk of hosting in another large company's data center, right? You own it all from top to bottom, and honestly if you have a small blog and you're using your ISPs connection, and you have this overkill of a Raspberry Pi cluster that is powering the static site, you're probably not going to ever get enough traffic to bring that thing down. You're probably fine.
TESS: Probably not.
TESS: Although I think Jeff's site is Drupal 8.
IVAN: Oh [laughing] so, not static, not static. Well, I'm very jealous of you getting to see that talk. That must’ve been pretty cool. I'm hoping that maybe we can get Jeff on the podcast to talk about his cluster and what he's been through and how it's evolved soon. So, Jeff if you happen to be listening, watch out for an email from us about that.
Let's talk about diversity at Flyover Camp. What did it look like? Were there the kind of usual cast of people that look like I do, white males, or what did that look like amongst attendees and speakers this year?
TESS: So, there certainly is a large contingent of white straight cis male people there as well. There were a lot of women there as well, and there were several POC as well. I didn't actually take any moment to really do any kind of headcounts on that. It just never crossed my mind to log that kind of information. But I did sit with several people which were really fascinating and really interesting to talk to, and that was really nice.
IVAN: I hope we can have more of that and more attention to that in the future and we'll try to continue to talk about it and bring it up in our podcast as well. What about attendance as a whole at Flyover Camp? Was it comparable to Drupaldelphia or to Twin Cities Drupal Camp? Did you get a feel for what it was like?
TESS: I think that it was more closer to the size of Twin Cities Drupal than Drupaldelphia. Drupaldelphia had a surprising amount of people in it. And it could have been complicated by the space, because it was a smaller space than Flyover Camp or Twin Cities Drupal, but there was certainly a large number of people there.
IVAN: Any particular sessions besides Jeff’s, that were memorable to you?
TESS: Oh geez, I'd have to look it over because so much of it was kind of a blur. I was kind of sad that I missed John Rearick's session about 45 Modules and Forty-Five Minutes. I caught the end of it. But, yeah, that would have been a really interesting talk to go to, because literally every slide has a timer. So, the talk is only 45 minutes long. So every slide is only a minute long.
IVAN: So, it's kind of like an ignite session, where it's 30 seconds per slide, 20 slides, something like that?
TESS: Mm hmm. I saw one by Ria Dixon called CloudWatch-ing, which was all about creating logs and alerts using AWS CloudWatch. That was really fascinating. And, it makes me wonder if there is a way to create similar mechanisms and use similar strategies in a purely open source implementation that doesn't rely on AWS's productized version of that.
IVAN: What is CloudWatch?
TESS: It's kind of an event and log tracking mechanism meant for distributed logging. There's a lot of that I didn't get into because it was mostly a case study about how they implemented it, and how they solved their own problems. There was a lot of additional research that I'd love to circle back to, but it was a really good session and I really enjoyed it.
IVAN: So, is CloudWatch then, kind of similar to Splunk?
TESS: I think that it's a bit similar to Splunk. I know that there might be part of that that's similar to Prometheus and Grafana which is a common Kubernetes logging mechanism.
IVAN: Yeah, Prometheus is pretty widespread as well, isn't it?
TESS: Mm hmm.
IVAN: Yeah. Okay. So, a couple of good sessions. Generally, you had a good time at the camp, gave two wonderful talks. Where was the camp? Was it at the University?
TESS: I believe it was. It was a pretty good location, although because it is in the middle of Missouri, I did have a problem getting to and from the Camp, because I didn't have a car rental. So I ended up walking there and that was a 20 minute walk in Missouri in June, which was a bit warmer than I’m used to. [laughing]
IVAN: Yeah, I guess the flipside of that is it could have been Missouri in December or January.
TESS: I mean I would have been fine with that but that’s me, I like winter. [laughing]
IVAN: [laughing] Well, go figure. I like it too. So, what do you know? Okay. So, the event venue was good. The attendance was comparable to TCDrupal. And before we wrap up, overall impression of the event? If there's another one next year are you going to go?
TESS: I would love to go again. It was a lot of fun to go there. And it’s a lot more interesting than I had expected it to be, which kind of lives up to the name. [laughing]
IVAN: [laughing] That's great. Well, I appreciate the time you spent with us today once again. Thank you so much for being with me. It's really been a pleasure.
TESS: Mm hmm.
IVAN: Well, Tess Flynn or socketwench, is the Devops Engineer here at TEN7, and she was just at Drupal Flyover Camp 2019, where she gave her talk, “Return of the Clustering Kubernetes for Drupal.” Of course, that's the third in a trilogy and the other talk, “Dr. Upal Is In - Healthcheck your Site.” Those slides are all online and a recording of the sessions are also available. Just visit this episode's webpage for those links. You’ve been listening to the TEN7 Podcast. Find us online at ten7.com/podcast. And if you have a second, do send us a message, we love hearing from you. Our email address is firstname.lastname@example.org. Until next time, this is Ivan Stegic. Thank you for listening.