avatar

Jiri Knesl

Posted on 11th July 2023

Clojure Corner with Sean Corfield (part 2)

news-paper Clojure | Interviews | News |

As we were talking to Sean Corfield, we decided to change from one corner to another. This time, it’s mostly Sean who is leading and Jiri answering.

Iain: There are lots of talk atm about how AI will change software development, ranging from “we’re all doomed” to “it’s the best thing since sliced bread”. How do you think it will affect us as Clojure developers specifically? Do you think we have any particular advantages or disadvantages?

Sean: Automation has always had an impact on the labor market but software development has, so far, remained fairly resilient to this. We’ve “automated” a lot of how we express solutions to problems, in terms of translating them into something that computers can execute, but the industry as a whole is still growing as software spreads into more and more devices and so we need more programmers today than ever before.

I think AI has the potential to automate some low-level aspects of what we do: at this point, it’s fairly good at generating boilerplate code, and it’s getting better at generating useful test code, so I think it will help eliminate some “grunt work” from what we do and allow us to focus on more interesting problems and solutions.

As Clojure developers, we’re already working further up the abstraction ladder so I think we are less “at risk” in terms of AI taking over our specific programmers’ jobs — but there is a risk that if AI gets good enough at generating solutions in mainstream languages like Java or JavaScript, companies may be less inclined to invest in non-mainstream languages. I think that is years away from happening: generative AI, based on language analysis, is great at producing “text” that often reads well but can still be full of errors — and it will require smart humans who understand the problem domain and the solution space to verify and correct those errors.

I think there are some potentially interesting applications for AI in our industry in terms of explaining legacy code — to help with maintenance — and generating (at least a first pass at) documentation. Perhaps we can also get some very “smart” linters out of this, identifying duplicated or unidiomatic code more “in the large”, as opposed to most of today’s linters operating “in the small”?

Iain: I have always been a fan of TDD and its variants, ever since I came across it in the Ruby world some years ago, but over the years I don’t think I have ever seen it make the transition into day-to-day use. Do you have any views on the practicalities of TDD? Should it play a bigger role in the development process?

Sean: I would say that I’m an advocate of TDD but I’m pragmatic about it. I think TDD is great when you have a good sense of what behavior you want at a fairly detailed level — so we follow that practice at work for new APIs, for bug fixes, and for a lot of new features in the frontend (React.js) application. For a lot of backend work with Clojure, we’re more likely to write Rich Comment Forms containing exploratory code as we figure out how best to manipulate data or how to structure data flows — and some of that will move into unit test code to prevent future regressions, or we might convert it to Rich Comment Tests and leave it in place https://github.com/matthewdowney/rich-comment-tests/

I think there are a couple of reasons why it hasn’t become more widely practiced:

  • Some TDD advocates are rather… zealous… about it and have said things like “If you don’t do TDD, you’re not being professional!” which has a tendency to turn people away, without even looking at the merits.
  • The whole idea of writing tests for everything upfront seems like a waste of effort to many developers (and to a lot of managers, unfortunately), when everyone is rushing to meet deadlines and cut as much of the process out as they dare.

Even many people I know who advocate for TDD now were initially very skeptical and really didn’t “get it” until they had forced themselves to use it for a while on a day-to-day basis.

Iain: You mentioned using Polylith in your current job. How has that helped in the development process? It’s particularly interesting that it is largely language agnostic, and appears to be addressing development at a high level of abstraction. I guess it also involves a certain level of discipline to use it fully. Would you say it has paid off? Perhaps this approach is one that is ready for wider adoption.

Sean: I think it’s important to understand that Polylith isn’t a library or a framework but more an architectural plan for organizing your code — and that’s why it can be applied to most languages, even tho’ there is only tool support for Clojure right now (and it is a very useful set of tooling).

A couple of things we have found very helpful with adopting Polylith:

  • Separating the “build” from the “application” — each artifact you produce can have its own build process and can determine its own dependencies, separate from the actual code that goes into it.
  • Discoverability — Because of the folder structure (bases, components, projects), and the consistent namespace structure, it’s very easy to see at a glance where functions come from.
  • Naming — the focus on small, self-contained, and hopefully reusable components means that you are more deliberate about how you choose your names, and it has actually encouraged us to discuss naming and code organization more as a team than we used to.

Some of the other things Polylith brings to the table have also been beneficial:

  • Incremental testing — the poly test tool can determine what parts of your system need to be tested based on what code has changed and what depends on that: that has cut down our CI times and saved us money (cloud costs).
  • Swappable implementations — Polylith lets you have multiple implementations for a component, based on the same API (interface in Polylith terms, but that’s a namespace with a set of public functions), and the implementations are selected at build-time (in projects), which has made it easier for us to support self-contained, lightweight applications that don’t use a database, as well as legacy applications that have to run on Java 8 (so we can isolate JVM-specific code in alternate implementations of a component and still maintain conformity of interface usage across our codebase).

Jiri: Have you seen modulith architecture? Do you like it? It’s a lot like component but usually more split than the usual app, db, jetty combo

Sean: I had not heard of that but reading up now it seems to sit somewhere between the monolith and the polylith level of granularity (and I see Spring’s modulith tooling supports the same sorts of module access checking that Polylith’s check tool provides!).

There are times when I’d like something a bit more coarse-grained than Polylith’s components but we’ve also found that breaking some obvious components along CQRS lines has had sometimes dramatic effects on our dependency graph and that it’s often the Command side of a component (or module) that brings in more dependencies and those can often be avoided — but that’s going well beyond just organizing code per business domain concepts most of the time.

Iain: and I’m running out of questions to ask Perhaps there is something you would like to ask us about Flexiana?

Sean: Yes, I’m curious to hear how Flexiana expanded geographically — you seem to have developers all over the world, but in fairly unusual places. Even the US branch is not somewhere people tend to associate with high-tech (Sheridan, WY)!

Jiri: Well, I started with Clojure about 11 years ago when I lived in the Czech Republic where I am from. Then, I moved to Spain and then to the USA. So it was quite certain we will be a fully remote company. And when I started Flexiana 8 years ago, I was in California, but the rest of the team was in the Czech Republic.

Then, after about 2 years, we hired the first dev from Poland, because I was seeing him here on Slack often. This was the moment we switched the whole company to English. I was still in the USA and the rest of the team was in Europe.

When we were using English, we could hire anywhere, but the main places were the EU and USA. The interesting thing is, I got a recommendation to start the US company in Wyoming because they told me it’s better for taxes. Then we hired employees in California, NY, and Texas and we had to file 4 tax returns and haven’t saved anything. If I incorporated Flexiana in any of these states, I would save. :smile:

Then, about 4 years ago, we moved back to Europe (to the UK) and we found that culturally, people from Latin America are very similar to us (by hiring Brazilian immigrants in Portugal). After that, we started hiring in Latin America and built a strong base there.

Today, we have just 1 person working in the USA, and half of us are in Europe, and the second half in LatAm. I must say, culturally, we’re very similar and I am very happy for this mix.

Sean: Re: filing four returns — yeah, the US tax system is ridiculous!! Trying to run healthcare for your company across state lines here is even worse!

Thanks for the history of how Flexiana has grown over nearly a decade — fascinating to see how a fully remote company can operate as it scales, and I totally get the “culture divide” between the USA and, well, most of the rest of the world, to be honest. Moving here from England was a serious culture shock and my (American) wife often jokes that we are two cultures separated by a common language.

Next time we are going to have Michiel Borkent in Clojure Corner.