With the system architecture more or less finalized, at least the first version of it, I now need to make a decision about the stack of the system.
Obviously some parts will be pure python: the classifiers are built on Pandas and Scikit but the rest of the system will probably need to be built on something more manageable: flask is not a good web framework and Django is probably overkill.
The candidates are Java/Spring, Scala/Play and Scala/ZIO. I’ve been thinking about diving into Scala again and now that Scala 3 is released I think it’s a good time to do that. Unfortunately, LightBend, mainteners of the Akka and Playframework, have been scaling down their support to Scala/Play and there’s no definitive answer of when Play will be upgraded to Scala 3, if ever. Granted, Play is a popular framework and there’s no way it will stay unmaintained for long but still, it’s not a good choice until things are clearer.
One new library that is getting very popular in the Scala world is ZIO, a functional effects library. ZIO is just a runtime and some data structures that allow you to package small pieces of code that do some kind of computation which you then can put together in ways that describe some business logic or some input that can be fed back into some other ZIO data structure. It’s effectivelly a way to describe programs by replacing control flow with data and functions. You don’t believe it when described but there are sample projects based on ZIO (and other packages from the ZIO ecosystem) that look extremely nice.
The last, and probably most boring, option is Java/Spring. The time of the large monolithic framework is over, in my opinion, people seem to be moving more and more towards building decoupled applications by glueing together small apps with infrastructure e.g. the app is the frontend that consumes disparate services or code running on a host, a container or even serverless. That been said Spring will remain popular in the foreseeable future just by sheer inertia if nothing else.
So far, ZIO with http4s is the only option that intrigues me. It’s a completely new paradigm and progress will be slow in the beginning but Scala 3 and the fact that ZIO has a lot of traction makes me optimistic.