Should we allow different technology stacks in a micro services landscape?April 11th, 2019 Software Architecture 'Handmade Ravioli' by Davide Ragusa on Unsplash
Don't worry, I'm not going to explain how to create ravioli. That's not my piece of cake. My daughters recognize my style of cooking by the boil-over fires. Some time ago I saw a caricature of comparing software architecture to Italian food, micro services were compared with ravioli. I was wondering whether or not we should have different technology stacks in a micro services landscape, so I wrote it down. Buon appetito!
Knowledge and experience
The world of tech is moving fast. It has been moving fast for decades, however it seems to me that it's going faster than ever. This might be due to the fact that the market is asking for full-stack, whatever that may be. I will dedicate a future post explaining why I think full-stack is bullshit. If you don't have readily available knowledge of at least a couple of dozen technologies you might feel not as effective as your colleges that do understand those technologies. This is already the case when you're only working in a single stack of technology. So I don't believe in developers being elite in multiple platforms. Of course there are exceptions that confirm this rule.
So within one developer you preferably specialize in one stack. If you're doing scrum you might be able to do some stacks with overlap in frontend technologies, but still being effective means you choose one technology stack for the team. And to be honest if you have multiple teams building software in your company, you would loose flexibility it moving members between teams if they work on different stacks of technology. Not that moving members between teams helps in scrum, but that's yet another story.
Micro services in an enterprise environment
Architecture in an enterprise is never build on a mono stack. One of the things you'll notice within enterprises is the large amount of different technologies that are used. This is partly caused by off-the-shelve products that have been purchased regardless of the technology they are build on and are customized to the needs. The other part is caused by the slow movement of changes within an organization. If you look at the past decades, we've seen things like Corba, Service-Oriented Architecture, Restful APIs and now Micro Services. It's not strange that we see some pieces of each in an enterprise organization. Those new architectures come faster than light!
But let's be honest, we understand choices like best-of-breed, but is it ideal? For those who never heard the term best-of-breed, it basically means choosing the software that matches the functional requirements best. This could mean the choice for SalesForce as the CRM system, while this would not necessarily fit in the infrastructure and technology knowledge within the company. This is a total different world compared to the past, SAP for everything, or Microsoft only environment. But still apart from the standard software there's enough custom build. I think it makes sense to choose one stack for the custom build software. So either go for Oracle Java, Microsoft .NET or Node JS, don't combine everything if there's no absolute need for this.
Intra micro service communication
Early evolutions of micro services quite often use Restful APIs to communicate to each other. It's good to build upon a standard, but to be honest, I'd rather both consume my Restful API and produce the Restful API in the same technology stack. Integration will be just easier. This is no different when using a message broker like RabbitMQ instead of APIs. Even though we serialize contracts to JSON and use AMQP as a protocol, using typed contracts and same message bus framework all around the landscape helps you focus on build functionality. In the past I've had quite a lot of trouble using Soap between Java and .NET, the WS-* specs still give me shivers. I will dedicate a future post explaining why I think intra micro service communication should mostly be based on messaging.
Best technology matching the functional and non-functional requirements
Sometimes you will see the need for a micro service that has some functional or non-functional requirements that can be implemented much easier in some different technology stack. These situations aren't even rare, quite often it's just a matter of using a different storage technology. Instead of using a regular RDBMS like SQL Server or Oracle you might see the need for key-value database like Redis or document database like Azure Cosmos DB. Somehow we accept something like that easier than suddenly adding Linux to the stack where Windows Server was the standard, or creating a very specific Restful API in NodeJS instead of the standard .NET Core based Web API. The truth is all have similar pain, it's not standard within the company yet. So that means not every developer has understanding of this new technology, yet. But more important this technology needs to live somewhere in production so the operation guys that maintain your production environment need to know how to run this correctly in production. Don't forget installing a MongoDB on your dev box is not the same as having it in a production environment where availability and security has different needs. You don't want to be the company where your MongoDB is held hostage.
I'm not saying you should always prevent to introduce new technology, but do it with care.
These were some thoughts that came to my mind. There are probably more viewpoints to this topic, but it's enough for now. There's no real answer to this question, I think. The only valid answer: It depends. So if you are in the situation where you think too much or too little technology is introduced within your micro services landscape, you have some of my viewpoints.
Software developers can make everythingMarch 28th, 2019 Software Engineer 'old hand tool collection' by Hunter Haley on Unsplash
Software development is a very creative profession. After gaining a few years of experience you will have a tool belt to create about anything people ask you. Where classical engineering leaves less room for creativity, because of rules on material properties and gravity, software engineering has endless ways to achieve a solution. For a large part that's why I love development work.
Can we really make anything?
I think we can. Of course there are things I can't make yet. This is about not being trained on the particular subject. Even for some of these subjects, I might not even be able to get trained because my head can't grasp the complexity in it. But that would be things like: Building software for a nuclear powerplant or quantum computing. For everyone those limits are different.
My point though, there's really a lot of things we can make as a developer. Sometimes we need to train ourselves to understand new technology, but having some basic experience makes any learning new language or library just a matter of time till until you will grasp it.
And if you want to exercise to build creativity in solving problems, you can try to do some kata's. If you're good in building programs in C#, but your TypeScript isn't fluent yet, do a couple of kata's in TypeScript to improve your skills.
Should we really make everything ourselves?
I still recall, the ages where we did .NET Framework 1.1, no I'm not talking about .NET Core, I'm really talking about 1.1. It must have been in 2007. I was assigned a project at an insurance company, we were going to build a insurance policy administration system. The project, being of the waterfall-type, actually got cancelled after about a year. But during the initial phase of the project we were looking for solutions for some technical challenges. One of them actually was logging.
How do you do logging today? You either choose the build-in logging framework in .NET Core, or use the build-in abstractions together with a well known framework, like serilog or NLog or even use the cloud with products like Azure Monitor or Stackify Retrace. During the before mentioned project however, there wasn't that much choice, there was log4net as a port of log4j and there was the Microsoft Enterprise Library which contained Logging and Instrumentation. When talking to the IT-architect, actually he wasn't much of a software architect at that time, but had more knowledge on infrastructure, he didn't like the Enterprise library because it was too slow (it actually was slow). The option to go with Open Source, log4net wasn't an option either. Wow, I was quite disappointed, the option we had to go for, was build-it-yourself. And even though it seems like the requirements to log towards both flat file and the EventLog sounds like an easy task. Wait for the incoming requirements where you'll have to add things like rolling log files, support for logging from multiple threads, and more. Actually at that moment I already told the architect we shouldn't do this ourselves, because it would be both hard and not our specialization. So we ended up creating a faulty log framework which luckily never ended up in production because the project got cancelled. Even though Open Source might not always be allowed as a choice, building it ourselves was a very bad choice. Don't build your own logging framework!
Some companies, like to hide every framework behind an abstraction of their own. I've seen a company write an abstraction in front of the Dynamics CRM Web Services, which leaned heavily on reflection, it prevented the need to generate a Web Service stub. End result? A mall-functioning abstraction which was overly slow and lost Integrated Security. I have also worked with remote team who abstracted everything, from logging to database persistence. I agree that there is some value in hiding the database access behind a properly implemented Repository and UnitOfWork pattern, however this was not the case. The team also wanted to do CQS but ended up with Commands returning query results. After lot's of talking we ended up removing those patterns and went with Entity Framework without further abstractions. This team, was not competent enough yet to add value with the before mentioned design patterns. Of course it's not always the case that abstractions are mall-functioning, but it still happens quite often that features are hidden which are actually very useful. Be always careful with abstractions. Ask yourself: Do they add value? Sometimes the insights about too much abstraction comes really late. I'm not sure if removing the too much from abstractions will be a wise business decision.
One of the other things I see every now and then, "We created a CMS ourselves, the options out there don't have the features we want.". I know some of the CMS solutions like Sitecore are quite expensive, but don't compare this with a scrum team of 5 which are building an inferior CMS solution for half a year: roughly € 50k each month, makes € 300k. How expensive was Sitecore again? If you actually purchased some parts of the software your time to market would be much faster. Make sure the teams are actually adding value to your processes within your company. Unless you're actually a supplier of software that is going to sell the CMS, don't build one yourself. Choose a solution and customize it instead.
Side note: This site is actually running on a custom build CMS. It's inferior to every existing free and paid alternative. Why do I build a CMS? I need a playground which I can use to gain experience and knowledge around certain architectural and design patterns, frameworks and tools. Besides that, I think it's fun to build it. However it's not a wise business decision when not counting the gained experience and knowledge which helps me sell myself to my customers.
Freelancing apart, togetherMarch 19th, 2019 Freelancing 'into the night' by Benjamin Davies on Unsplash
Freelancing can be quite lonely at times. Looking for a project and having trouble finding one immediately will make you think: Is there something wrong with the market? Is my knowledge mismatching with the asked experiences? Is my rate too high? When you have to decide which insurances you need to have, questions popup: Do I really need a professional liability insurance when I'm working in a scrum team? Should I worry about defaulters?
Of course there's a reason you started freelancing, but in the end you're not alone in this world. More people are freelancing, and they might have similar questions. So let's find ways to help each other.
Freelancers on-site at the customer
When you're working on-site at your customer, the chance is very high you're not the only freelancer being hired. If you're lucky even your scrum team has one or more freelancers. However that doesn't even matter, just to talk to them at the coffee corner and have lunch together. You'll learn from each other. Everyone does their business different and that's totally okay, but maybe they inspire you in doing something aside your main business.
One of the things I noticed in the past years, that freelancers tend to remember you, when everybody moved on to new projects. Even last week I was approached by a former freelance college who said there would be an opening at his customer in a couple of weeks.
Form an informal group
Once I started freelancing, I already knew a some freelancers. One of them, Michel van Duijse, is a former team mate at a former customer, more than 10 years ago. We discussed about the things we would miss as a freelancer if we would stop working for a consulting company. One of them being, the monthly/bi-monthly pizza-sessions where college's would share their new gained knowledge. Eventually it would result in, Freece a group of freelancers he started together with other freelancers he worked with on projects. When I started freelancing he asked me to join Freece as well. Freece is an informal group of 9 freelancers that meet bi-monthly. We eat something together, and one of us gives a technical presentation, to share knowledge. Besides that we also have a closed slack hub where we share projects that we can't fulfill ourselves and help each other out with business related issues. I think a group like this works best with a group of around 8-10 freelancers. Even though it's quite informal, everyone should try add an equal amount of value to the group.
Some thought that might help you form an informal group as well:
- Only allow freelancers in your group that at least of one of the group members have worked with on a project
- Take a fixed day that suits everybody
- Agree upon who will give a presentation and who will arrange the location (diner and presentation room)
- Legal arrangement aren't necessary
- After the night, share the invoices of diner and location (arranged by the person who arranged the location)
Network at meetups
Every now and then, I join a free meetup. During a meetup there are most of the time one or two technical sessions presented, but before and after the sessions there's time to network with each other. I know, as a techie it's not that easy to network. If you find this hard you should just try to talk to one person you don't know yet. This kind of evenings help getting your knowledge up-to-date and makes your network larger.
Some organizations like dotNed and .NET Zuid organize Microsoft .NET related events at sponsored locations. Some other companies organize their own tech events, like 4dotnet and iSense do in The Netherlands.
So, freelancing apart, together
Yes I know some of the readers will tell me, networking with other freelancers won't help you. Instead networking with potential clients or with recruiters does make sense. I'm not telling you to only network with other freelancers, I'm telling you to also network with freelancers.
A lot of time I'm contacted by recruiters who ask for my availability, which I'm not most of the time. However 4 out of 5 situations I can recommend a freelancer that will possibly meet the recruiters requirements. This happens the other way around as well, other freelancers know a position will be available in a couple of weeks and they tell me.
So continue freelancing apart, together.
Time for a sunriseMarch 11th, 2019 Life 'silhouette of trees at golden hour' by Jaime Serrano on Unsplash
After years of no blogging, it's time to start writing again. I've had two previous periods of blogging. Looking back at those periods a lot has changed, again.
From 2003 till 2006 I wrote my blogposts in Dutch. Those blogposts are lost in history, I can't recall the topics I wrote about. This doesn't matter, there was no evergreen content in there anyway.
On the 1st of July 2007 I restarted blogging, this time it was on mark.mymonster.nl. I wrote about Life and Technology, but mostly it was just development related blogposts. After 2013 my writing became irregular, with only a small number of posts in 2014 and 2015. And then there was silence, a lot of silence.
I'm seeing this silence as a long night. But the sun is rising. I'm back!
When I restarted blogging in 2007, I was just about to get married. During the coming years our family would grow from my wife and I to include three beautiful daughters.
These days my girls are doing great! They are now aged 10, 8 and 5 years old, and growing up really fast. My oldest daughter is going to secondary next year with her year-and-a-half younger sister following soon after that. My youngest daughter is taking swimming lessons while her bigger sisters play hockey.
My wife started working again about a year-and-half ago after being home for our girls for over 4 years. She is doing great, working as a teacher for children with multiple handicaps. I'm really proud of the work she is doing for those children.
A lot of technology has come since I restarted blogging in 2007. Things like Silverlight, Windows Phone, Windows 8, all gone now. Actually it was my interest in Silverlight and Windows Phone that kept me going on the blog for quite some time. Microsoft even recognized me by awarding the precious Microsoft MPV Award. Those were very interesting days, besides having tight contact with Microsoft I really loved the community around Silverlight and Windows Phone. Still miss those days.
Even though things come and go, .NET has been a steady one in my career. These days it's mostly .NET Core and Azure. This combined with new architecture patterns like Micro Services gives me energy to start blogging about my experiences with .NET Core and Azure.
In late 2015 the company I was working for went for bankruptcy. This had some impact on my life. Sometimes you need some forces from outside to push you into the right direction. For me this direction was to start freelancing fulltime. Even though I am working onsite for my customers, things are quite different. For a starters I needed to build a network to find customers who wanted to hire me. Next I needed to make sure I got my finances right. Making sure I could pay the mortgage and the income tax while feeding my family.
This is working out great so far. But now I am thinking about how I can adjust my business to earn passive income next to working by the hour which I still love to do. I want to share my explorations on this path to a changed freedom.
Rise and shine
Enough about what happened and the plans. Let's start to execute them!