“R-S-A protected configuration provider?! What the f#$k is that, and why the f#$k is it dying an inglorious death while taking my CM with it?!” Those words, friends, are mine. It’s a rare issue that can make me angry enough to club a baby seal silently whisper expletives at an inanimate object, but it’s Monday. And I’ve had coffee… lots of coffee. Y’see, I’ve recently been involved in upgrading an entire organization’s mission-critical servers from 2011 to 2013. For the most part, as we all hope in such circumstances, it’s been a breeze; nothing to set the pulse aflutter. Until today. And that terrible, miserable, unhelpful exception provided by ASP.NET.
This does have a solution, and it’s a pretty easy one at that. I could just write a paragraph with the fix, and be done. But would we learn anything?
Why does it happen?
Tridion’s primary configuration file stores a wealth of sensitive information, and to prevent that information being tampered with it’s encrypted. Database connections, application settings, and all sorts of Tridion’s nethers could be left exposed to rubbernecks and Nosy Nicks, so Tridion makes use of the RsaProtectedConfigurationProvider to encrypt and decrypt entire sections of its configuration file. If you open Tridion’s core configuration file (the path is %Tridion%\config\Tridion.ContentManager.config) you can see that most of the file is actually hidden behind these little encrypted walls that are only opened when the Content Manager is up and running, and only by IIS.
Those long, seemingly random strings of characters? Those keys are used by IIS to unlock that configuration section, which is pretty cool until Monday-and-coffee-fueled rage. Solid encryption isn’t just about having a key to a particular door, it’s also about that door having a lock for the key. When the Tridion CM is installed (or upgraded, as the case may be) part of its function is to have IIS generate matching sets of keys and locks. The keys sit in the configuration file, and IIS maintains a key store to authenticate those keys whenever they’re used, so that the lock can be properly used.
Ordinarily, a typical Tridion install to a single machine isn’t going to unearth this issue, but upgrading load-balanced instances of the CM will if you simply copy-pasta the configuration file from server to server. Load-balancing the Content Manager is, by my experience, a fairly common practice. It’s tempting to take shortcuts when making upgrades because you want the two servers to be in parity. Two servers sharing the exact same configuration file? Tridion will serve you this:
So I gotta reinstall the Content Manager executable?
Nope. There are legitimate infrastructure cases that require the sharing of configuration files, and the process is surprisingly easy (I’m not thanking you, Microsoft; this is just one pain point of the many that you owe me for). Multiple servers can share the same duplicated configuration file providing that they also share the same key store. To explain the process I’m going to refer to the first server as Avocado; this chap has a working Tridion CM, and is where the configuration file was copied from. I’m going to refer to the second server as Banana; this guy is delivering that terrible message above, and is the recipient of a duplicate Avocado configuration file. Note: you’ll need Remote Desktop access to both servers for these steps.
- RDP into Avocado, and open a Command Prompt as administrator.
- At the prompt navigate to c:\windows\microsoft.net\framework\v4.0.30319.
- Type the following, and press Enter:
aspnet_regiis -px "TridionRsaKeyContainer" "c:\keys.xml" -pri
- You should receive a success message, and if you take a look at the root of your C: drive you’ll notice you have a fancy new keys.xml file. Great!
- RDP into Banana, and again open a Command Prompt as administrator.
- Copy-pasta that shiny keys.xml file from Avocado to the same spot on Banana, so you have keys.xml at the root of your C: drive.
- Delete keys.xml from Avocado.
- At the Command Prompt on Banana, again navigate to c:\windows\microsoft.net\framework\v4.0.30319.
- Type the following, and press Enter:
aspnet_regiis -pi "TridionRsaKeyContainer" "c:\keys.xml"
- You’ll receive another success message, and at this time delete keys.xml from Banana.
- Restart the Tridion Content Manager services, restart IIS, and you should now have a fully functional Tridion CM in the browser. No more fugly and mysterious error message.
Wait… what just happened?
If you look at the top few lines of Tridion.ContentManager.config you can see a configuration section called “configProtectedData,” under which is a single RsaProtectedConfigurationProvider instance with a name of TridionRsaProtectedConfigurationProvider, hence the Tridion-centric nature of the original error message. This provider is what IIS uses to encrypt and decrypt all of the other configuration sections in the file. The provider also contains a key piece of information that we need, which is the name of the key store that IIS uses to authenticate attempts to access all that sensitive information. In this case we’re looking at the keyContainerName attribute, and it has a value of “TridionRsaKeyContainer”.
The first command we issued on Avocado was essentially telling IIS to spit out the key store named “TridionRsaKeyContainer,” which has the good keys that both servers are trying to use, but only exist on Avocado. IIS spit out the key store as an XML file. We then copied that XML file to Banana, which has non-matching keys compared to the configuration file in use. The command we issued on Banana imported Avocado’s keys into Banana’s IIS key store. We restarted Tridion’s services – along with IIS – in order to ensure that Tridion would reinitialize using the updated keys, and presto!
Nice post Rob. You might want to try alcohol-free coffee for a while though!. I just wanted to take you to task for your sweeping statement: “a typical Tridion install to a single machine isn’t going to unearth this issue”.
Sadly – it’s very likely that you’ll also have similar trouble even with a fairly simple installation, because the key-store is associated with the user account that reads or writes the data. If you ever want to update encrypted configuration values, you’ll need to be logged in as the same account that was used to do the install. (Now go away and imagine a few scenarios where that is fairly unlikely or impossible.) The good news is that the fix is just as simple. You can use the same tools to export and import keys from one account to another. As long as you’ve thought about it in advance, that is…..
As a well-behaved Tridion Infrastructure consultant you would naturally ensure that post-install the TridionRsaKeyContainer is made accessible to all administrators by giving admins full control to it (as opposed to what the installer does: give MTSUser/Network Service read and the installing user full control and everyone else no access). Simple as running
aspnet_regiis -pa “TridionRsaKeyContainer” “ThisPC\Administrators” -full