Every Sitecore developer and enthusiast should by now have already heard about Sitecore Commerce or better Sitecore XC. This product is already existing for about 2 years now and has run through big transformations. From the old days, where Sitecore Commerce meant using a Microsoft Commerce Server connected to Sitecore via Commerce Connect; through the beginning of the transformation where Sitecore introduced the revolutionary Commerce Engine, which was initially introduced and worked in some kind of hybrid mode with the old Microsoft Commerce Server; till nowadays, where Sitecore Commerce is called Sitecore XC and only consists of the new Commerce Engine without any need to have the Microsoft Commerce Server.
Within this little article I would like to tell a story about this journey, from a developers perspective, which means, how were the developers confronted with the new technology, what were various pitfalls and how you were able to overcome these pitfalls. Of course the focus will be on the experiences with the very latest version of Sitecore XC (9.0 Update 3). The aim is, to give all of you an idea, what it means to get confronted with the task to implement a Sitecore XC solution and hopefully to give you some ideas, how to avoid pitfalls to make your life easier with some personal best practices in the end.
From what I heard, at least from the German community, and maybe also other communities from all over the world, Sitecore XC is still some kind of magic, which terrifies even self-confessed Sitecore developers. But kidding aside, Sitecore XC is quiet a huge topic with various new architectural concepts, frameworks and ways how to implement a solution. Hopefully with this article I can calm some of you down and show that Sitecore XC is magic, which can be unboxed and mastered.
The daily business situation
Let’s assume you are now a Sitecore developer and some of your customers ask you, if they should use Sitecore XC for there next commerce project. What can you do now to advise the customer professionally and convince him to use Sitecoere XC. And a even better question, how can you, as a developer, get quickly into this black box topic Sitecore XC .
I myself was facing exactly this problem some time ago and to be honest I am still learning today and most probably will never stop learning and experiencing new aspects of Sitecore XC. So my first advice is: Never stop learning. Even if you think, you know everything, I can promise you, there will be customer requirements or situations, where you have to dig even deeper into the system and learn new aspects of Sitecore XC.
Where to start
The next question is, where to start learning, if you never have been confronted with Sitecore XC. In the very beginning of Sitecore Commerce 8.2.1 there very nearly no documentations available, except of the official Siteocore documentation. My approach then to learn more about Sitecore Commerce was
- Read the Release Notes / Sitecore Documentation
- Install the Demo Storefront
- Play around with the OOTB features to get a rough overview
After doing these three steps I had a good overview, what Sitecore Commerce means, where the strength are and how to use it. Of course you can use the same approach for every new topic and newer versions of Sitecore XC. What I personally always do, when a new Sitecore XC release was published, I do all the three steps again. I read the release notes to get informed about the newest features, I install the newest demo storefront and play around with the new features.
Note: Nowadays you are in a even better situation, than all of us 2 years ago. Nowadays Sitecore made enormous progress in the product itself as well as the whole documentation. In addition we have a great community, which helps with answers and blogposts, Sitecore itself provided really cool training (100, 200, 300) to get into the topic commerce, slack, stackexchange and much more. You can see a great list of everything here.
How to go on
The next question is, how could be go on from that point? To read a lot of documentation and play around with the demo storefront is nice, but you will not get practical experiences in working with the product, in this case Sitecore XC. But in my opinion, that’s exactly what you need. In the documentation and demo storefront you will always see the system from it’s best side, so the best case scenario, where everything works perfectly fine. But what happens, if we go on from that point? What happens, if be implement our first functionality? How will the system react on real business scenarios? Will it be stable? Can it handle that? All these questions should come into your mind at this point. And the only way, how we can find answers to all of these questions is: Try it out! Start implementing your custom logic. Think of common use cases or scenarios and implement them to see what happens. I can promise you, you will come to the point, where something breaks, or does not work the way you expect it to work.
What I did e.g. was to think of the use case to use Sitecore XC for a German customer. After having a look at the features, I quickly found out, that the OOTB functionality for calculating taxes was build the way, that you can set one specific tax rate environment wide. Unfortunately in Germany and other countries, we have the situation, where we need multiple tax rate per environment, which vary from product to product. So this was the best point to start, writing a custom plugin to see, if I can overcome that lack of functionality with a custom tax plugin.
How to implement functionalities in Commerce Engine?
But here we run into another issue. If you start working with Sitecore XC you will see, that you will not only get a new module or new feature for Sitecore XP, like with the installation of WFFM (Yes I know it’s dead, but a good example 😀 – RIP WFFM), but a whole new architectural concept, with an independent solution based on .NET Core and how to implement pieces of logic for Sitecore. In Sitecore XP e.g. you build pages and modules most likely following the Helix principles. You will always have something like controllers, which use services to do something and get view models to be displayed on views. If you need some kind of settings, you most likely use setting configs under app_config or settings items, spoken very rough on high level.
But if you start on Sitecore XC you will quickly find out, that most of these things are not working this way. And because of that, I think it makes the learning curve even harder. To understand the fundamentals of Commerce Engine implementation takes a lot of time. Most likely you cannot just open the solution and start coding. Most probably then your results won’t be what you want to implement. For me personally it was a real changeover to get into these new architecture of Commerce Engine. After some time, training, reading and testing it became better and nowadays we also have a strong community, which shares also knowledge about how to implement stuff in commerce engine.
From all these sources, daily work and experiences you could derive some Best Practices or DOs and DON’Ts for working with commerce engine and implementing plugins, which might help working with Sitecore XC. Why I just picked this list is because I’ve already come across all these things in my day-to-day work.
Some Best Practices
A Plugin in Commerce Engine is not a Feature in Helix architecture
When you previously worked with Sitecore XP and standard module implementation, like implementing a new image-text module, a stage, or some fancy list, slider or map, you will always have the same approach of doing that. And if you follow some basic architectural guideline, like Helix, you have even more a standard approach of doing such work. When you now start working on Sitecore XC and implement your first feature or better plugin in commerce engine, it happens quickly, that you apply the same architectural guidelines also in commerce engine. When you read the documentation and some blog post, you know about, how a plugin project is structured, and what components belong to a standard commerce engine plugin. The issue, I observed now in some implementations is, that developer handled commerce engine plugins as some kind of feature layer projects within Helix architecture. What does that? It means, that, when you for example have plugin A, which consists of some new Pipelines and Components to extend e.g. your cart and you have Plugin B, which uses the components from Plugin A, the approach was to take out these components from Plugin A, create a new Plugin C, which holds these and many more components from many more Plugins and let reference Plugin A and B to that Plugin C. This approach is quiet familiar in Sitecore XP, where you have two feature layer projects and create a foundation layer project, which both features are allowed to reference. You can see some diagram, which illustrates that below.Why is that a bad approach? It is, because it tears the plugin apart completely. Your plugin is then spread over two or even more different plugins, which at least violates the basic principle of Opt-In-Complexity. You cannot just include the stuff / plugins you really need and want into your solution anymore, because it is spread all over your solution over various dlls. Removing all of these plugins from your solution, would cause also other plugins to not work properly anymore, which means all the plugins have to stay active, even if you don’t need them. And if you e.g. need just one component in that magic Plugin C from before within your new Plugin, you are obligated to reference the Plugin to get the component, but most likely get even more garbage you don’t really want. So if you have such a scenario with Plugin A and B, just let Plugin B reference Plugin A and let it use the component from there. Some basic principle could be, that a plugin should contain ALL pieces of logic, which belong to the Plugin and should not be spitted up among various VS projects. A small illustration is added below.This is also a standard approach in OOTB provided plugins by Sitecore itself, see the diagram below. You see, that Orders plugin for example just references the Carts plugin to use components, Pipelines and more from Carts to provide special Orders functionality.
CommandsController / Minions / CommerceCommands should contain as less code as possible
If you implement a new functionality in commerce engine within you plugin you should take care, that the CommandsController.cs, the Minions and the CommerceCommands should not do all the work. What I mean is, that e.g. if you have a minion, which grabs some information from external systems, which should be used for all of your orders, you should better implement a custom pipeline with blocks, which do the actual work. One benefit of using a pipeline is, that is is completely adaptable by other plugins by adding or removing various blocks to the pipeline. Another great benefit of using pipelines is, that the actual work is placed at one single location, which then can be used by different pieces of code. So for example you can then execute your pipeline from a Minion in regular intervals, from web requests via CommandsController or by custom CommerceCommand, used by service proxy, without unnecessarily duplicating the logic.
Use Service Proxy if possible instead of implementing web requests on your own
Most of you already know different ways to interact with commerce engine from within Sitecore XP instance. You know, you can call the commerce engine directly or use Commerce Connect API. The benefit of using Commerce Connect API basically is, that Commerce Connect includes various analytics functionalities, which are triggered and collected OOTB.
But, what if you have to extend or individualize the commerce engine. If you do that, most likely Commerce Connect is not usable anymore. In such cases you will have to implement a direct communication with commerce engine. Also this can be done at least in two ways. You can implement web requests and execute them on your own. An example of that can be seen below.
In such an approach you have to take care of the URL to call, all headers, parameters and also deserialize the result on your own.
Another quiet nice approach of interacting directly with the commerce engine is to use a service proxy. Because commerce engine uses OData it exposes all its functionality in a way, that clients like Sitecore XP can easily generate a powerful service proxy. Sitecore also provided us with a VS solution to do that. With such a service proxy, you can easily access the whole commerce engine in a common and easy way. Examples how to do that, you can see in the implementation of Commerce Connect API seen below.
Always evaluate policies on runtime, when you need them
If you implement a custom plugin and need some settings, commerce engine allows you to do that, with the usage of policies. From one of my previous blog posts you already know, what a policy is.
A policy is a named, versionable and variable set of data that can be used as facts within behaviors to influence behavioral outcomes. This provides an auditable mechanism for viewing, simulating, and changing core decision criteria that might be used by business processes or other policy-driven behavior. Various plugins have specialized policies to add additional facts to existing policies and/or new policies, or to implement new concepts, such as dynamic pricing.
So, what I now saw, was the approach to use a block in the IStartNodePipeline, which just grabs the policy by context and saves that to a static instance. Every future usage of policies now grabs the policy values from that static instance, instead of always getting the polices by context on runtime.
But what is wrong with that approach?
The problem with that approach is, that the IStartNodePipeline is called with a special environment. And of course the policy values of that environment are grabbed and stored within the static instance. When we now call some functionality of our plugin from another environment, than the IStartNodePipeline was called, we will get the wrong policy values from the static instance. So at this point, we are not able to properly have multiple environments within our commerce engine, just because we used the standard mechanism of policies incorrectly.
So, if you use policies within your plugin, always use the current context to grab the right policy values on runtime.
Do not change the status of an order “manually” within your plugin
How do you change the status of commerce engine orders? What I already saw was a simple approach of doing that. The situation was, that some external system gave feedback on the order status based on the fact, if the order was shipped or not, or if something went wrong etc. The engine now gets the feedback within a defined minion. The minion determined based on the feedback, the next status of the order, grabbed the order and directly changed the status. The problem with that approach is, that commerce engine has it’s own way of changing the stati of orders. Within commerce engine there exist special minions like, PendingOrdersMinion, WaitingForAvailabilityOrdersMinion and some more, which in the end scan dedicated lists of order, like all pending orders, grab them and execute a corresponding pipeline, like PendingOrdersMinionPipeline. Within these pipeline some blocks check various conditions if and how an order should be proceeded and set to a new status. But that’s not all. These blocks also manage commerce entity lists, to remove this order from the previous list and put it correctly into the next one, take care of indexing and more.
So if you want to set the status of an order, just add the information for that on the order, implement your custom block in the right pipeline and let the Minions do the job. Also if you plan to create new order stati, just follow the standard approach of creating a dedicated minion for that, commerce entity list to hold all the orders of your status, extend the existing pipelines with blocks, which put the orders into your status and create your own pipeline, which handles orders in your status.
Do not hard code storefront or project specific stuff within your plugins
Imagine the situation, that you implement a new plugin, which should be able to react differently if called from different storefronts. What you now should not do is something like seen in the screen below.
But why should you not implement stuff like this within a plugin?
If you implement such statements within your plugin, it would be closely coupled to your Sitecore XP instance, which uses that plugin. Such a plugin with hard coded storefronts, or other hard coded properties, cannot be used in other instances anymore. But this is a core architectural idea of plugins. Plugins should be implemented in a way, that they can be shared among a various number of instances e.g. like a Nuget package, without changing a single line of code. And yes, it is possible. At least, if you are faced with the given problem.
What you should do, if you want to implement different behavior when used by different storefronts, is add for every storefront a custom environment. Use policies to implement individual settings for your plugin. Based on the storefront the corresponding environment with custom set of policies can be grabbed on runtime. Now you can implement some kind of generic logic, which uses your policy to decide, which way to go. Such an approach is then dependent on the environment, which can vary from instance to instance. But the code can stay always the same.
Let Commerce Engine calculate all the pricing of carts, orders and so on
This point might be obvious. But I already came across pieces of code, where commerce specific functionality was implemented within Sitecore XP. Image you have the requirement, that an order can only be created, when the cart has reached a minimum order value. What you could do now, is to grab the cart from commerce engine and within your Sitecore XP cart feature just check the value of the cart against a predefined threshold, which is stored in some settings item. Based on the check, you could allow the user to proceed the checkout or not.
Of course this approach works well. But from my point of you it is not optimal. Some kind of minimum order value is a very commerce specific feature. Such commerce specific feature should then also be handled by commerce engine on its own. What you should get from the commerce engine is the information for your current cart, if the minimum order value has been reached and maybe also what the minimum order value is. Depending on the specific use case, the determination, if that value has been reached can be quiet complicated, because you maybe not only want to check against Subtotal or Grandtotal, but against Grandtotal – FulfillmentFee to exclude shipping costs from that value. And at least if we reach the point, where complex calculations are necessary, these should stay in commerce engine as single source of truth. And image the case, where you have multiple endpoints, which interact with the commerce engine, like Sitecore, a mobile app, Postman and more. And you implement such kind of minimum order value in Sitecore directly, all the other endpoints would have to implement the same logic, so the behavior is consistent. Otherwise it would be possible e.g. that Sitecore can only create orders with a minimum order value of 50 Euro, but the mobile app or Postman can already create orders with any value, which might violate business requirements.
What you could do is to create a new plugin. Within this plugin you create a new pipeline block, which hooks in into e.g. GetCartPipeline at the very end. Within your new block you now do your check and calculation. The threshold can be saved in a dedicated policy, so it also can be different per environment. After the calculation you could enrich the cart with a new threshold component, which holds the information if the threshold has been reached or not and maybe also the value of the threshold to display that in frontend. In addition you could also add a new block in CreateOrderPipeline, which just checks the new flag, if the threshold has been reached, and if not just forbid the order creation. This way the responsibility of calculation of commerce specific features stays in commerce engine at on single location, where it belongs to. And from that point on it does not matter anymore, from where you try to proceed with the checkout, cause every endpoint will get the same results from commerce engine and can just react on that, instead of doing all the calculations.
If you want to expose functionalities in Commerce Engine, use Pipelines instead of Services, Repositories, Factories and so on
In standard Sitecore XP implementations, where you follow a micro-services approach, in implementation of new features, you most likely create services, which can be consumed by others. An illustrative example of that approach can be seen in the implementation of new SxA modules, like described in the following article.
This approach works well in Sitecore XP. But if you start implementing features / plugins in Sitecore XC this approach is less optimal for the given architecture of Sitecore XC. A core principle of Commerce Engine is the concept of pipelines. Benefits of such kind of approach are
- Pipelines can be adapted the way business requirements require it
- You can individually add, remove, replace blocks within pipelines
- You can create complete new pipelines, which then can also be adapted by other plugins
- You can remove pipelines / plugins completely for opt-in complexity
- There is a special log file NodeConfiguration, which holds information about all existing pipeline and containing blocks to easily work with them
- and many more
If you would now just create a service within your plugin, it would be much harder for other plugin to find and use that service, others than with pipelines. And it would be even impossible to adapt that existing service to fit on individual business needs. Therefore we would violate and lose some important core principles of commerce engine by extensively using services over pipelines.
If possible, use commerce entity lists for performance reasons
Image the requirement, that you have implemented a Minion, which has to check e.g. if Orders are in a specific state, and if so, do something with these orders. What you could do now, is to use FindEntityPipeline for every single order to grab the order and afterwards check the status and only keep orders, which are in the right state.
But what you also could do is, to use OOTB feature of commerce engine; so called entity lists. Sitecore provides us already with some predefined entity list, especially for order handling. Sitecore holds e.g. a list of all pending, on hold, problem and many more orders. If you now want to check the status of an order, you just have to read the corresponding list and check if your order is in. If so, you can afterwards grab the order from the system and do your stuff. This way we won’t grab any order unnecessarily, which is in a wrong state.
Such entity lists can also be created and maintained on your own for your custom business needs. Depending on your use case, you just have to add your commerce entity to a new list by adding a special TransientListMembershipsComponent to the entity and add the name of the commerce list, like seen below.
This way commerc engine will create a new entity list, if not existing, and puts this entity in there. After that, you are able e.g. with a minion to grab all entities of that new list again. If you want to remove your entities again from the entity lists, you just have to use RemoveListEntitiesPipeline, like seen below.
Add new Pipelines in the standard way of using SitecorePipelinesConfigBuilder
In general, if you you want to create a new pipeline, you just have to create that pipeline and add it within the SitecorePipelinesConfigBuilder within your ConfigureSitecore.cs class.
But I have already seen another approach of creating a pipeline. In that approach, you have a static manager class, which holds in the end a list of pipeline definitions. If you now want to add a new pipeline, you just have to access that static manager class and add your pipeline by definition. This definition can implemented as an enum. So these pipelines would only exist within that manager class virtually and is not accessible from outside
Of course this approach works and in the end we get our pipelines and all of these pipelines are usable via the manager class. But it also has some drawbacks
- You can only add pipelines of dedicated input and output parameter, based on your manager class
- These pipelines do not appear in the NodeConfiguration log file, so you have no chance of having a look, which pipelines are configured and how
- If you try to adapt some pipeline, which was created this way, you have to reflector the existing code to see, which pipeline exist and how they are configured, which is nearly impossible
So my recommendation at this point is, not to over engineer stuff or reinvent the wheel and lose all of the advantages, if you could use a standard approach.
Use standard functionalities as often as you can, instead of reinventing the wheel
If you are working with Commerce Engine you will come to the point, that you have to extend standard functionalities with new ones. One use case, which I have already seen, I would like to show case to you. The task was to implement a new fulfillment method. The approach of doing that was to create a new interface in commerce engine to add that fulfillment method, a new component, which just inherited from standard Component and a piece of logic, which grabbed the current cart to add that new component.
But what are possible problems with such an approach?
With such an approach you would in the end implement a workaround of using OOTB existing fulfillment functionality. Sitecore already has powerful fulfillment capabilities with Commerce Connect integration, dedicated components and pieces of logic to handle order creation and other stuff afterwards. If you now just use your own component and treat it like your fulfillment component, you would break all the functionalities, which are already existing and you have to implement all of them again on your own. In addition some related features like pricing or promotions would also not work the way you expect anymore. Assume, that you have added your custom fulfillment component with a fulfillment fee and you would like to use a promotion, which removes the fulfillment fee from your cart as benefit. This promotion would not work anymore and you would also have to implement something new on your own.
A better approach of implementing e.g. a new fulfillment option is to use all the OOTB components. First you would just add that new method as Commerce Connect fulfillment Item in Sitecore XP. You would also add that method within standard fulfillment policy. If you need a new component with new special properties for your new fulfillment option, just create a new component and inherit from standard fulfillment component and use that component on your cart. This way Sitecore also recoginzes OOTB your new component e.g. as standard fulfillment component. And all the related functionalities work again as expected. Therefore, if possible, try to use standard components and extend these functionalities with custom business needs, instead of reinventing the wheel.
Use Roles for scalability in Sitecore XC
Lets have a look at a standard setup in Sitecore XP and, how we could integrate Commerce Engine into that environment. A very simple approach would be the following one.
We have one CM and on CD Sitecore XP web server and just one single Commerce Services instance, which serves the traffic from all the Sitecore instances. Of course even in small setups, this approach would be insufficient. Cause just having just one single instance, which serves traffic from authors from CM system, shoppers from storefronts and executing scheduled tasks in the form of minions in the background is most likely not enough.
A simple approach of handling that is to just increase the number of Commerce Services conected with a load balancer to handle more traffic. A diagram of how this could look like can be seen below.
Of course, instead of using dedicated servers connected via load balances, you could also place the commerce services closer to the web servers, if you want.
But this is not the final approach of how scalability is done for Sitecore XC. Sitecoere XC uses the concept of Roles to realize scalability. From one of my previous blog posts, you should already know, what a Role is. Here is a part of the definition again to bring it back to your mind.
Role: “The Sitecore XC solution implements the concept of Commerce Engine roles to support scalability. In a production environment, traffic is usually split up among multiple installed instances of the Commerce Engine, which are usually physically located close to their traffic sources… “
So in the end we have multiple Commerce Services, which are located close to the web servers. The special thing now is, that each commerce service has a different logical roles, based on its location in the environment. Physically all the Commerce Services are identical, but each has its own set of policies depending on the traffic, which should be served. OOTB Commerce Services can have the following roles
- Authoring – Close to CM
- Shops – Close to CD
- Minion – Decoupled
- DevOps – Close to CM
A diagram, of how this architecture could look like, can be seen below.
With such an approach, all the single Commerce Services only have to serve very specific traffic and only have to do very specific work. And every single instance can be configured in an optimal way to do that.
If you adapt existing or implement new Pipelines, use NodeConfiguration logfile to see the current state of your changes
If you follow the core principle of commerce engine and work with pipelines you will most probably adapt existing pipelines or create complete new ones.
The question now, when you want to adapt existing pipelines is, which pipelines exist in commerce engine and how are they currently configured?
The answer to that question is simple. Sitecore provides us with a new type of configuration file. under /wwwroot/logs/. There you find configs which start with NodeConfiguration, like seen in the screen below.
This logfile holds information about all the currently configured pipelines with used blocks. This file lists all the existing pipelines, with input and output parameters. In addition it lists all the blocks, which are used by the pipeline, each also with the information about input and output parameters. All that can be seen below with the example of IAddCartLinePipeline. These information are essential for every developer, who tries to inject custom blocks into such a pipeline.
If you add a new block, you of course have to take, that the input parameter of your block matches the output parameter of the previous block and your output parameter matches the input parameter of the next block. And of course within such a log file you are able to see, if your custom block or whole custom pipeline is added to the commerce engine or not.
If you find bugs in the system, don’t hesitate to create Sitecore Support tickets
Only if we report bugs and misbehavior of the system to Sitecore, they are able to fix such stuff and make the product more stable. When I started working on Sitecore Commerce 8.2.1, it really felt like a Beta test for me. I tried out as many features as possible and played around with the product. While doing that, I experienced a lot of issue, which I thought might be bugs. So for every issue I created a single Sitecore support ticket. You can see an extract of the support portal below.
In the end it turned out, that most of the issues really were bugs in the new commerce engine. One version later, most of the reported bugs were bugs by Sitecore itself. So by reporting misbehavior back to Sitecore, they were able to make the product more stable, which is, in my opinion, worth it to invest some time in ticket creation and reporting.
Within this article you got insight in some of my experiences from the last 2 years while working with Sitecore Commerce 8.2.1 up to working with Sitecore XC (9.0.3). I saw, how everything started with the introduction of new commerce engine, evolved and what we got till now. It is true, that starting with the big topic Sitecore XC as developer is not easy. But hopefully with this article, you see, that not everything is such magic, as it seems to be in the beginning and you got some personal best practices, which might help in the future.