Heads up: some issue I faced during recent CSM monitoring implementation. Working on remediation with Microsoft. Still, beware.
While working on one of the recent consulting projects I faced a strange behavior dealing with CSM monitoring implementation. This article is written based on the hands on experience.
As you know on 64-bit boxes in app pool configuration you can check to “Enable 32-Bit Applications” option. Say, for some reasons I had to enable it for CSMCollector app pool (“OperationsManagerCsmCollector v4.0”):
Having done this I noticed that I started missing CSM data. Albeit the web application pages are injected with uX scripts, the requests are sent to CSMCollector (I can check this with Fiddler), but I get no client-side events and statistics received at all.
I enabled APM trace log on affected server and got plenty of “CSMCollector ServiceManager is not active” errors like below:
Agent version: 7.0.5000.0
.NET Framework: v4.0.30319.269
ProcessID: 30488; InstanceID: 0;
InstanceName: w3wp.exe;
ApplicationPool: OperationsManagerCsmCollector v4.0;
Full Domain name: /LM/W3SVC/8/ROOT/CSMCollector-1-129917338260256050;
[0]30488.21608::09/10/2012-02:57:03.248 [Agent.CSMCollectorService] []
[Error] :CsmCollectorService.ProcessCsmEvent{csmcollectorservice_cs257}( 015DF0E0 )CSMCollector ServiceManager is not active.
[8]30488.1648::09/10/2012-02:58:34.369 [Agent.CSMCollectorService] []
[Error] :CsmCollectorService.ProcessCsmEvent{csmcollectorservice_cs257}( 0171C5DD )CSMCollector ServiceManager is not active.
[0]30488.48032::09/10/2012-02:59:42.669 [Agent.CSMCollectorService] []
[Error] :CsmCollectorService.ProcessCsmEvent{csmcollectorservice_cs257}( 02ABE64D )CSMCollector ServiceManager is not active.
[1]4844.19088::09/10/2012-04:34:09.540 [Microsoft.EnterpriseManagement.OperationsManager.Apm.Common] [] [Error] :CSYSConsoleBase.Print{basesysconsole_cs644}( 010D5FB2 )An error has occurred. Please contact Technical Support if this issue persists.
Type: Agent.Common.CSYSConsoleBase+AgentException
Message: Unexpected result:
‘setupPath != null’.
Source: Microsoft.EnterpriseManagement.OperationsManager.Apm.Common
StackTrace: at Agent.Common.CSYSConsoleBase.CheckLogic(Boolean logicExpression, String predicate, ExpCategory category)
at Agent.Collector.ServiceManager..ctor()
End of stackTrace: at Agent.Common.CSYSConsoleBase.GetExeptionInformation(Exception exp)
at Agent.Common.CSYSConsoleBase.Print(Exception e, String messageStr, String parameter, EntryType entryType)
at Agent.Common.CSYSConsoleBase.Print(Exception e, String messageStr, EntryType entryType)
at Agent.Common.CSYSConsoleBase.PrintFromExternalMethod(Exception exp, String message, EntryType entryType)
at Agent.Common.CSYSConsoleBase.PrintFromExternalMethod(Exception exp, EntryType entryType)
at Agent.Collector.ServiceManager..ctor()
at Agent.Collector.ServiceManager.Nested..cctor()
at Agent.Collector.ServiceManager.get_Instance()
at Agent.Collector.Engine.Suppliers.StatisticsReportingHttpHandler.ProcessRequest(HttpContext context)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
at System.Web.HttpApplication.PipelineStepManager.ResumeSteps(Exception error)
at System.Web.HttpApplication.BeginProcessRequestNotification(HttpContext context, AsyncCallback cb)
at System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context)
at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr managedHttpContext,
IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)
at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr managedHttpContext,
IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)
at System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr pHandler,
RequestNotificationStatus& notificationStatus)
at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr managedHttpContext,
IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)
at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr managedHttpContext,
IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)
Agent version: 7.0.5000.0
.NET Framework: v4.0.30319.269
ProcessID: 4844; InstanceID: 0;
InstanceName: w3wp.exe;
ApplicationPool: OperationsManagerCsmCollector v4.0;
Full Domain name: /LM/W3SVC/8/ROOT/CSMCollector-1-129917396546536050;
[10]4844.38608::09/10/2012-04:34:27.827 [Agent.CSMCollectorService] []
[Error] :CsmCollectorService.ProcessCsmEvent{csmcollectorservice_cs257}( 0398CA93 )CSMCollector ServiceManager is not active.
[8]4844.6724::09/10/2012-04:41:28.379 [Agent.CSMCollectorService] []
[Error] :CsmCollectorService.ProcessCsmEvent{csmcollectorservice_cs257}( 020CF7FB )CSMCollector ServiceManager is not active.
Hm.. Looks like a bug. When I switch back and disable 32-bit apps for CSMCollector app pool, everything works pretty well.
According to log most likely the problem deals with CSMCollector installation key search in registry. When you enable 32-bit applications on the pool, it might start looking for CSMCollector keys in 32-bit branch and cannot find it there, while actually it’s registered in 64-bit registry path.
Still no resolution or workaround found for the issue. Probably it can be resolved only with code fix.
Regards!