How Does vCenter Operations Manager Collects Data from a vSphere Environment?

This article is to ensure that I document a discussion I had a few days back with some Gurus of vCenter Operations Manager. The discussion ignited from a customer requirement where they wanted to know:
“If they can ETL (Extract, Transform & Load) 2 years old data from the vCenter Server DB into their freshly installed vCenter Operations Manager Environment.”

Needless to say, customers see a great value in the Performance Analytic and Capacity Planning engine of vCenter Operations Manager. They want to ensure that they can use the historical data from VC and get some intelligence from the vCOps about the performance and capacity trends in their infrastructure in the past. Alongside, giving a peek into the history, this can also become a strong historical base for vCOps to provide future predictions which is another strong area of the product.
The requirement above immediately raises a few questions about the process of data collection:-

  • Does vCOps talk to vCenter Database through the vCenter Adapter to collect the data? If yes, can I feed historical data to give vCOps a kick-start?
  • What is the impact of Data Collection on vCenter Database? Do I need to increase resources CPU/Memory/IOPS to ensure that the performance is fine?
  • Other questions are around Data Collection cycles, remote collection etc.
Well, to be honest there can be unique questions from each deployment. Therefore, instead of looking at the questions, let’s look at the process of data collection.
To recap, let us first look at the architecture of vCOps vAPP which will give us a clear picture as to where does the collection happens. (I wrote an article a few months back about the architecture & sizing of vCOps in case you are interested.)
If you look at the red highlighted circle, you can see that the collector sits on the Analytics VM, from where is uses adapters to collect data from resources such as vCenter, vCloud Director, Configuration manager or even 3rd party sources.
Now if we look at the collection process specifically for the vCenter Adapter, this is how the data is collected:-
vCenter Operations Data Collection Process using vCenter Adapter

Well this explains how data is collected from a vSphere Environment by vCenter Operations Manager. Since this model does not talk to vCenter DB, it is quite clear that you cannot have an ETL approach which I discussed in the beginning of this article

To conclude the data gathering process of vCenter Operations Manager is completely through PerformanceManagerAPI where vCenter Server acts as a proxy and the data is pulled from the ESXi hosts.

Share and Spread the Knowledge. 

Published by Sunny Dua

Product Management Leader with 18 years of experience with Digital Transformation, Application Modernization, Artificial Intelligence, Machine Learning, Business Observability and AIOps. Led large product portfolios at VMware and now at AppDynamics. Passionate about defining vision, strategy and executing on complex product roadmaps to build successful and innovative products. A Stanford Graduate School of Business LEADer with interests in the field of Critical & Analytical Thinking, Strategic leadership and Design Thinking.

One thought on “How Does vCenter Operations Manager Collects Data from a vSphere Environment?

  1. VMware vCenter™ Log Insight™ might be able to analyze the historical log datahttp://www.vmware.com/products/datacenter-virtualization/vcenter-log-insight/features.html

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: