21Sep, 2023
Cutting a Useragent's Value When It Exceeds the Maximum Possible Length for xDB Databases
It's taken years of managing Sitecore sites for this one to come across my desk, but apparently there's no protection for the UserAgent value being longer than the maximum allowed length when saving the User's session to the xDB. Though it's fixed since version 10, there's no patch for earlier instances, so we're going to have to add our own pipeline processor to cut the offending UserAgent values short.
The issue arises when the Sitecore.Analytics.Pipelines.SubmitSessionContext.SaveDevice processor runs, which commits the current session's information to the xDB. During this action, it takes the UserAgent value, which was set during the Sitecore.Analytics.Pipelines.CreateVisits.InitializeWithRequestData processor. In here, an interaction is created with data such as:
- BrowserVersion
- BrowserMajorName
- BrowserMinorName
- ScreenWidth
- ScreenHeight
- OperatingSystem
- Ip
- DeviceId
- Context Site
- UserAgent
The last property is the one we're talking about today, and the one giving us troubles! When this string exceeds 900 characters, exceptions like the following would be seen in the logs:
Cannot set column '[UserAgent]'. The value violates the MaxLength limit of this column.
System.ArgumentException:
at System.Data.DataColumn.CheckMaxLength […]
at Sitecore.Xdb.Collection.Data.SqlServer.Builders.StagingTableBuilder`1.BuildTable […]
at Sitecore.Xdb.Collection.Data.SqlServer.Managers.SqlDataRecordsManager […]
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw […]
That's a pretty obvious error, and it might be possible to check the value being assigned to the UserAgent when the initialize is called, but I know the only issue is during the save action to the database, so we'll manage the fix there.
Adding Protection With a New Processor
So, we want to check then shorten the UserAgent if it's longer than 900 characters. Realistically, it would never be this long, so it's edge case and I'm ok with shortening it. It's that, or the error, or we don't save them at all. And we want to save them!
The following is the config file used to add the new processor. It's pretty wordy, but just copy the whole thing and do what you want with the namespaces. You can see this adds the new processor “ValidateUserAgent” just before the first processor of “SaveDevice” in the SubmitSessionContext section.
<configuration>
<sitecore>
<pipelines>
<submitsessioncontext>
<processor type="SitecoreFundamentals.Pipelines.SubmitSessionContext.ValidateUserAgent,
Sitecore.Foundation.SitecoreExtensions" patch:before="processor[@type='Sitecore.Analytics.Pipelines.SubmitSessionContext.SaveDevice,
Sitecore.Analytics']" resolve="true">
</processor>
</submitsessioncontext>
</pipelines>
<settings>
<setting name="SitecoreFundamentals.Pipelines.SubmitSessionContext.ValidateUserAgent.MaxUserAgentLength" value="900"></setting>
</settings>
</sitecore>
</configuration><span></span>
Next is the processor itself, which isn't really complicated at all. After some brief null checks, the maximum allowed length is retrieved from a standard configuration setting just in case someone would want it changed.
The next step is to see if the UserAgent is too long, and then shorten it. The logging is left at Info level because this should be pretty rare, and you'd want to see an entry in Production if this happened.
namespace SitecoreFundamentals.Pipelines.SubmitSessionContext
{
public class ValidateUserAgent : SubmitSessionContextProcessor
{
public override void Process(SubmitSessionContextArgs args)
{
ValidatorExtensions.IsNotNull<SubmitSessionContextArgs>
(Condition.Requires<SubmitSessionContextArgs>
(args, nameof(args)));
var interaction = args.Session?.Interaction;
if (interaction == null || string.IsNullOrWhiteSpace(interaction.UserAgent))
return;
var maxUserAgentLength = Configuration.Settings.GetIntSetting("SitecoreFundamentals.Pipelines.SubmitSessionContext.ValidateUserAgent.MaxUserAgentLength", 900);
if (interaction.UserAgent.Length > maxUserAgentLength)
{
Sitecore.Diagnostics.Log.Info($"ValidateUserAgent -> Trimming UserAgent as it is longer than the allowed length of {maxUserAgentLength} ({interaction.UserAgent.Length} characters). Was: {interaction.UserAgent}", this);
interaction.UserAgent = interaction.UserAgent.Substring(0, maxUserAgentLength);
}
}
}
}
Will This Be Addressed for Future Releases?
This has been already reported and registered as a missing functionality. A fix was implemented in Sitecore 10, and further reading can be found here.
