Remix In Production at Tesco - An Interview With Hugo Jobling - Part II/II

Remix has been out for a while proving its worth in the trenches. Now's a good time to gather some evidence whether it lives up to its promises or not.

Hey Full Context Developer

(I’m writing in a personal quality, opinions are mine and are not reflective of Tesco’s)

This is Part 2 of my interview with Hugo Jobling a Principal SWE and colleague of mine at Tesco.

You can find Part 1 here where we talked about his career, what it’s like to develop software for this company and his experiences using React for over a decade.

In this final part we discuss his opinion about Remix, that he signed up for as soon as it went public - for a license fee back then - and used ever since. We also share some insights about an app it powers in nearly 2k Tesco stores in the UK. Let’s jump into it!

🎛️ Remix

- Learnings from buying the license at day one to using it in production

Joe: I think it’s fair to say that early adoption of the right technology can open the door to rare and valuable career options. It definitely paid out for you. But you also bet on Remix very early. Can you tell me a little bit about how that happened?

Hugo: I guess part of it is just like cultivating taste. It’s the same as with movies or music. There are people who have good taste and then they recommend things that are also good, right?

As you start to watch more good movies, you start to have an appreciation more of what is a good movie and what isn't. I think you can do the same thing with technologies.

Joe: That’s a really interesting analogy I haven’t heard before but thinking about it, it makes a lot of sense!

Hugo: Or take directors for example. There are certain directors who I know, if they make a film, it's probably going to be a decent film. The same thing happens in the JavaScript space. After a period of time, you start to hear the same names come out again and again. A few years ago Pete Hunt was a name you would hear about in the React space or Dan Abramov or Kent C. Dodds.

Joe: Yeah, all the classic React names, I’m very familiar with them too. So would you agree if I sum this up as “To cultivate a great tech taste all you have to do is follow the right people”?

Hugo: That sounds about right. One of the things that resulted in me making an early bet on Remix, is it's Ryan for a start! Ryan Florence is a known quantity in the React space. And actually, he's another one of those characters who I first encountered in the Ember community, and then he moved across to the React space. I think this is another one of those interesting tech envy things, right?

The React frameworks now are doing stuff that Ember was doing 10 years ago. The idea of routing and data fetching as first-class concepts is not new. That was around back in the Backbone days. And we're just rediscovering that those things were a good idea before and they're a good idea now.

I saw Ryan and Michael be like, hey, we're going to make a framework that's based on some of the concepts of how we've been thinking about React Router. You can buy an early adopter license for $200, and you'll get a hoodie. And I was like, early adopter license, day one, give it to me. So I signed up, started playing with it. And while using it, I was like, you know what? This has some good ideas. Having built and worked with large scale React applications for years, you realize, as everybody realizes, that it's called Create React App, it's not called Maintain React App. And there's a reason for that, right?

Joe: That one always cracks me up. It’s spot on!

Hugo: Yeah, there comes a certain point where that won't take you any further. There was a big period of time where your next step was custom Webpack config. There was no good next natural evolution. And then there was a period of time after that where for a certain class of application, there was a really good answer for that problem. And it was Next.js.

But the problem with Next.js that I found was that it wanted you to be building content websites.

So you're building Hulu, it's really good for Hulu because the primitives they needed were hooked into the framework. Next.js was really good for doing a single data fetch to load up the page and then display a bunch of stuff on it. But where it always fell down was, now I've got some user generated data on the application and I need to get that back to the server.

Next.js did not really have that good a story for it so you end up having to build custom API routes to be able to post data back to the server. Then you get into this whole mess of, OK, now I've got to pick a form library, and I've got to build an API route for every single page I've built.

There's a mismatch between how I'm getting the data onto the page and where it's going once it's on the page. So what I really liked about the conceptual model and the philosophy behind Remix was that they saw that gap. They realized that a huge proportion of web applications are mostly tables and forms. So they said: let's build a really good story behind tables and forms.

And then they went one little step further as well. In the last few years, the web platform has come up with good primitives for how you do this stuff. So you have the Fetch API, you have a capital-R Request, capital-R Response. And so we can build a framework around those core primitives that in addition to having a good answer to how do I get my form fields back to my database, doesn’t make you learn another framework’s own way of doing things. You're just using the stuff you already know. You already know fetch, Remix simply gives you a way to hook it into your application.

Joe: These are some really important observations and very good advice for people looking to adopt Remix. I’ve written about the framework extensively on my previous blog and my take was very similar.

Remix is best fit for web apps centered around users not only fetching but interacting with data that lives in a database. That’s the use-case where it can really shine.

Hugo: So again, it's a trade-off. If you're building Google Docs, you have to deal with the fact that two people on the other side of the world need to be able to edit the document at the same time. In order to get the performance, reliability etc… you needed, you had to make your back end much more complicated. And you had to invest in having a bigger team of people. If you are building that type of product, you're going to have to bite that bullet. But if you're not building that application, if you're building for example CRM systems internally at companies you're doing just basic stuff, or you're building a point-of-sale system that lives on the same physical device as the APIs, you have a different set of constraints. And so picking a technology designed to solve somebody else's problem makes you bite off all the complexity without reaping the benefits because you don't need those benefits.

The reason that I look at tools like Remix for our problem space is that its baseline is assuming you don't have all these Facebook or Google scale problems. It has more of a progressive enhancement approach. What the baseline experience assumes is, just load all the data to the page at the top of the page and render the entire page.

And then if it turns out, hey, that's too slow, Remix has the escape hatch, the concept of deferred data where if you can say, these 3 bits of data need to be loaded for the page to render, but this one can wait. Then the page can get that as a Promise, put it in a Suspense component. And then it works hand in hand with the built-in concepts in React. It's just a thing that React already supports and gives you the way to show a loading spinner for a period of time after the page is loaded. You can build that, but the default is not that every part of the experience has all these complications.

With Remix you only get the complexity where you need the complexity.

When I worked with Apollo GraphQL type applications, I was looking at a page and I had to ask. How did the data get on this page? Where does that come from? I have to look at every component on the page to find out what components specify the data it needed. Whereas, when you go to a Remix codebase, I just find the URL then I find the root component that matches that URL. That's where the data came from. Very conceptually simple. Another can of worms is async behavior inside of a useEffect which absolutely sucks. It's a nightmare to work with. And this is why, again, I like the Remix data fetching because it pushes that stuff outside of React, into a loader which is just normal JavaScript async/await code. And so you don't have to worry about useEffect and all of that nonsense inside. You can have nice clean components.

I tend to think: I'm not that smart. I want things to be simple, and I want the tools to make me look smarter than I am. When I work on a Remix application, I'm really productive because when I'm following the happy path, the pit of success, it pushes you along like a tailwind.

Joe: That’s the feeling we all want to get from the tools we use! Do you think that’s why others are adopting Remix as well?

Hugo: I would be surprised if not! But in a sense it’s also a part of the evolution of React. We are expanding the types of applications that are easy to build through frameworks. Take a business that’s, say, building a company blog and they've already got React developers. So having a React framework that lets them work on this is really useful. For that, maybe Next.js is good enough. Then they need the CMS to be built. You could have WordPress in headless mode, or you could use Ghost in headless mode but we've already got these React developers. Wouldn't it be good if we had the same skill sets applied to all of those? That’s where Remix comes in. So I think that's part of what is pushing this. We already have these skills. Let's expand the set of problems I can solve with the skills I already have.

We're not taking a bigger slice of the pie, we're making the pie bigger.

Joe: Thinking about Remix this way just gave me that warm Christmas type of sensation for a second. But now that we’ve talked about its upsides I’m also really interested in the situations where it’s not a great fit! The only serious problem I’ve run into was when I tried to take some state coming from the Remix API, put it into some real client-side state-management solution like Redux, work with it for a while and then wanting to sync it back to the Remix server. I’ve never pulled it off properly.

Hugo: I mean, the question is like, why would you?

Joe: I was simply doing it out of technical curiosity but there are many application in the Level 3 Web Complexity Category like the web version of Figma or Excel where you simply don’t want to have every little state change to get propagated to the server but still need a scalable state management solution in-browser.

Hugo: Yeah, Remix typically seems to want the server to be the source of truth for data in your app. It's not looking to help you build an application that has a lot of client-side state. There's a small subset of problems where you do actually need a single page application because you need to be doing real-time processing, right?

🙋‍♀️ RAP

- The Remix app that runs in almost 2k Tesco stores

Joe: Hmm, that made me curious, doesn’t the RAP fit that category? I know it’s a Remix application running in the Tesco stores working with real-time data. Can you introduce it to us and share how you tackled this issue there?

Hugo: So the RAP stands for Remote Attendant Person. It’s the application that runs on a dedicated machine in places where we have the self-service tills that lets a Tesco employee look at the state of the tills in one place. They can see things like, oh, this person needs me to go over and approve an age approval, or this till got an error and the printer needs to be fixed or whatever. And the employees can take action remotely like, if a person at the till is obviously 50 years old, I can just hit approve on the over 16 age approval without having to walk all the way over. So it's connected to the tills in real-time through individual web socket connections, and gets a feed of all the events happening on them, which then causes it to update its own state. The RAP and the tills have to sync together because if I approve an age approval on the RAP, it needs to disappear on the till. If I approve it on the till, it needs to disappear on the RAP.

Joe: That’s a great overview and now it’s more clear to me what might cause some problems with the Remix architecture for this use-case. So if the RAP app has all these real-time connections to the till machines in the client, it needs to react to the incoming events in-browser as well. Sounds like a classic SPA thing with client side rendering but then how does it benefit from the SSR capabilities and actions/loaders that give Remix a lot of its powers?

Hugo: The first version of the RAP I built was loading the list of tills in the background, render that client-side, and then have each of those independently create its own WebSocket connection to the back-end. That worked, but then you had the thing you were describing, which is like, why am I even using Remix at this point? All it's doing is giving me ESBuild and a page to render. The way I solved that problem was to move the real-time data to the server and have that push it to the client. It's much easier to trust the server to be the source of truth especially because they're only events, there is no state managed on the server. We manage the state locally. And that's done using a state machine library called XState because then we get the benefits of that it's actually deterministic. I can be sure that I've handled every event correctly or I've handled all the error cases correctly.

In the way the RAP is architected the whole app is a deterministic state picture that's based on events coming in and it just spits out a deterministic state. There's an EventSource that the server uses to send messages to the front end and then the UI just renders the corresponding new state.

Joe: In a sense, that doesn’t sound too Remix-y to me.

Hugo: I don't know how much they would promote it but with Remix you can have a real-time data feed on the API routes using Event Source. And then you just fire Server Sent Events at that, and it dumps a stream of messages out to the client side. You can make that as sophisticated or not as you want. It's just a text-based protocol. So in the RAP, the way we're doing the live updates with that is a JSON protocol where we send new states line by line. So every time the server updates, we send a new state to the application. It supports named messages so you could say, update, and then an update payload, delete, and a delete payload. And with that you can have a more complex mapping to say these messages go to this handler and these messages go to that handler. Then the interaction model, if you want to do it that way goes like, a form submission mutates some data, and then the update gets pushed back to you through the EventSource rather than it being the response.

Joe: This sounds quite nice as a concept. A clean mapping of events to state to UI going from the machines through the WebSockets to the Remix server to the application ending up in XState and rendered by React. How does it do in the real-world?

Hugo: There's no active development going on it but the fact that it didn’t need a release in the last 6 months is a testament to its stability. It's pretty reliable. One of our Engineering Directors the other day was saying, oh, I had a look at the code base. Seems really nice and neat and reasonable. So I think that's good. I mean, you should have a look at it yourself and see what you think.

Joe: I will for sure, I’m super excited to check the codebase of a live Remix application. I think this is a good place to (w)RAP things up. Thanks Hugo for sharing all your insights with us. It was an exciting conversation!

✍️ Author’s Note: Since the interview I’ve done so and I really liked the project. I also had a few interesting internal discussions with other developers involved with it and the generic sentiment is very positive about the RAP codebase. (In case if you wondered). But Tesco is a huge company and I came to know about other Remix projects running in our estate. Let me know if you want me to cover some of those too, or if you would like to see similar interviews in the future!