User Tools

Site Tools


extensions:vsphere-data-collector

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
extensions:vsphere-data-collector [2018/05/30 09:01]
bruno
extensions:vsphere-data-collector [2019/04/23 12:19]
vdumas
Line 1: Line 1:
-====== ​Vsphere ​Data Collector ​======+====== Data collector for vSphere ​======
 ---- dataentry summary ---- ---- dataentry summary ----
-name             : ​vSphere ​Data Collector ​+name             : Data collector for vSphere 
 +description_wiki : Collector ​for vSphere data synchronization in iTop CMDB (VM, Hypervisor, Cluster)
 index_hidden ​    : yes index_hidden ​    : yes
-version ​         : 1.0.7 +version ​         : 1.0.11 
-release_dt ​      : ​2017-04-24 +release_dt ​      : ​2018-12-31 
-description_wiki : PHP application to collect inventory information from vSphere +itop-version-min :  
-itop-versions ​   : 2.0 +download_url ​    : http://​www.combodo.com/​itop-extensions/​vsphere-data-collector-1.0.11-413.zip
-keyword_tags ​    : inventory, data synchronization,​ vmware, vsphere +
-dependencies_s ​  : itop-config-mgmt +
-author ​          Combodo +
-download_url ​    : http://​www.combodo.com/​itop-extensions/​vsphere-data-collector-1.0.7-218.zip+
 code             : combodo-data-collector-for-vsphere code             : combodo-data-collector-for-vsphere
 +state            : stable
 +product_hidden ​  : not-included
 +alias-code_hidden : vsphere-data-collector
 +alternate-name ​  : vSphere Data Collector
 +module-lists_hidden : itop-data-collector-base/​1.0.13,​ itop-data-collector-vsphere/​1.0.11
 +diffusion ​       : iTop Hub
 ---- ----
 +
 <​related_components>​Other versions of this component:</​related_components>​ <​related_components>​Other versions of this component:</​related_components>​
  
Line 18: Line 22:
 This stand-alone application collects information about a whole data center from a vSphere server (using the vSphere Web Services) and synchronizes this information with iTop using several Synchronization Data Sources. This stand-alone application collects information about a whole data center from a vSphere server (using the vSphere Web Services) and synchronizes this information with iTop using several Synchronization Data Sources.
  
-{{ :​extensions:​vsphere-data-collector.png?​nolink |vSphere Data Collector connections}}+{{ :​extensions:​vsphere-data-collector.png?​nolink&​450 ​|vSphere Data Collector connections}}
  
  
Line 24: Line 28:
  
   * Automated inventory of Servers (with their Brand, Model), Hypervisors (with their OS Family and OS Version), Farms, Virtual Machines (with their OS Family and OS Version).   * Automated inventory of Servers (with their Brand, Model), Hypervisors (with their OS Family and OS Version), Farms, Virtual Machines (with their OS Family and OS Version).
 +  * Optional inventory of IPv4 addresses and logical interfaces
   * The collector can reside on any system with web access to both vSphere Web Services and iTop.   * The collector can reside on any system with web access to both vSphere Web Services and iTop.
   * Automatic creation and update of the Synchronization Data Sources in iTop.   * Automatic creation and update of the Synchronization Data Sources in iTop.
  
 ===== Revision History ===== ===== Revision History =====
-^  ​Version ​ ​^  ​Release Date  ​^ ​ Comments ​ ^ +^  ​Release Date  ​^  ​Version ​ ​^ ​ Comments ​ ^ 
-|  1.0. ​| ​ 2017-04-24 ​ | Fixed the value for the status of VMs: '​production'​ instead of '​active'​. | +|  2018-12-31  ​|  1.0.11 | Corrects regression introduced by 1.0.10 :\\ * Improved support of iTop 2.4+ (obsolescence flag)\\ * Removed a debug trace\\ * Fix Virtual machines which "​connectionState"​ is not "​connected"​ are skipped\\ | 
-|  1.0.6  ​|  2017-03-08 ​ | - Code cleanup.\\ - Data Synchro ''​status''​ and ''​full_load_interval''​ are now homogeneous and configurable parameters for all data sources.\\ - OS Version and OS Family have their own mapping table.\\ - Fix for the Hypervisor status value (now defaults to "​production"​).\\ -  Addition of the ''​check_soap.php''​ script for troubleshooting.\\ -  Protect against a PHP crash in case of a disconnected ESX.\\ - Do not collect an IPV6 as the management IP address of a virtual machine (since the field can only contain an IPV4). Many thanks to Andrew Armstrong for proposing this fix.| + 2018-08-24 ​ |  1.0.10 | Handles TeemIp :\\ - Automatically detects if TeemIp (as an iTop module or as a standalone application) is present.\\ - Optional synchronization of IPv4 addresses\\ - Optional synchronisation of logical interfaces | 
-|  1.0.5  ​|  2016-12-08 ​ | New options to bypass SSL certificate validation and automatic detection of invalid certificates. | +|  2018-07-02 ​ |  1.0.9  | Robustness :\\ - more traces in debug mode (--console_log_level=9)\\ - skipping virtual machines which "​connectionState"​ is not "​connected",​ since most of their properties are not accessible and would crash PHP/Soap (with a core dump !!) | 
-|  1.0.4  ​|  2015-11-16 ​ | Replace line-breaks by spaces inside the description of a VM. Fix the +1 on the number of vCPUs | +|  2018-02-22 ​ |  1.0.8  | The ''​os_version_mapping''​ was used only for Hypervisors,​ made it work for VirtualMachines also.\\ Proper support of UTF-8 characters in regular expressions (the code now always uses the ''​u''​ modifier) | 
-|  1.0.3  ​| ​ 2015-04-17 ​ | Added the collection of the "​description"​ field for a VM | +|  2017-04-24 ​ ​| ​ 1.0.7  | Fixed the value for the status of VMs: '​production'​ instead of '​active'​. | 
-|  1.0.2  ​|  2015-02-23 ​ | Servers must be loaded before hypervisors | +|  2017-03-08 ​ ​| ​ 1.0.6  | - Code cleanup.\\ - Data Synchro ''​status''​ and ''​full_load_interval''​ are now homogeneous and configurable parameters for all data sources.\\ - OS Version and OS Family have their own mapping table.\\ - Fix for the Hypervisor status value (now defaults to "​production"​).\\ -  Addition of the ''​check_soap.php''​ script for troubleshooting.\\ -  Protect against a PHP crash in case of a disconnected ESX.\\ - Do not collect an IPV6 as the management IP address of a virtual machine (since the field can only contain an IPV4). Many thanks to Andrew Armstrong for proposing this fix.| 
-|  1.0.1  ​|  2015-01-07  ​| Beta version | +|  2016-12-08 ​ ​| ​ 1.0.5  | New options to bypass SSL certificate validation and automatic detection of invalid certificates. | 
-|  1.0. ​| ​ 2014-05-13 ​ | First alpha version |+|  2015-11-16 ​ ​| ​ 1.0.4  | Replace line-breaks by spaces inside the description of a VM. Fix the +1 on the number of vCPUs | 
 +|  2015-04-17  ​|  1.0.3  | Added the collection of the "​description"​ field for a VM | 
 +|  2015-02-23 ​ ​| ​ 1.0.2  | Servers must be loaded before hypervisors | 
 +|  2015-01-07 ​ |  1.0. | Beta version | 
 +|  2014-05-13 ​ ​| ​ 1.0.0  | First alpha version |
  
 ===== Limitations ===== ===== Limitations =====
  
-This version of the collector does not collect the Data Stores (they do not exist in the iTop standard data model) ​and the Interfaces.+This version of the collector does not collect the Data Stores (they do not exist in the iTop standard data model). Logical interfaces may be collected in the case where TeemIp (module or standalone) is used. 
 + 
 +If IP collection is enabled, only IPv4 addresses are collected, not IPv6.
  
 The current version of the collector is designed to collect information from a **single vSphere** server. To collect and reconcile information form several vSphere servers into one iTop, check the section [[vsphere-data-collector#​collecting_information_from_several_vsphere_servers|Collecting from several vSphere servers]] below. ​ The current version of the collector is designed to collect information from a **single vSphere** server. To collect and reconcile information form several vSphere servers into one iTop, check the section [[vsphere-data-collector#​collecting_information_from_several_vsphere_servers|Collecting from several vSphere servers]] below. ​
Line 89: Line 100:
  
 ^ Parameter ^ Meaning ^ Sample value ^ ^ Parameter ^ Meaning ^ Sample value ^
-| synchro_user | If the user account used for running this synchronization is //not// an Administrator,​ then its login must be specified here, since iTop allows only the administrators and the specified user to run the synchronization. ||+| synchro_user | If the user account used for running this synchronization is //not// an Administrator,​ then its login must be specified here, since iTop allows only the administrators and the specified user to run the synchronization. | |
  
 ==== Placeholders in the configuration of the data sources ==== ==== Placeholders in the configuration of the data sources ====
Line 107: Line 118:
 </​code>​ </​code>​
  
 +^ Parameter ^ Meaning ^ Sample value ^
 | full_load_interval | The delay (expressed in seconds) between two complete imports of the data. The objects which have not been detected by the collector during a timespan longer than this interval will be considered as obsolete and marked as such in iTop. Adjust this value depending on the scheduling recurrence. | 604800 | | full_load_interval | The delay (expressed in seconds) between two complete imports of the data. The objects which have not been detected by the collector during a timespan longer than this interval will be considered as obsolete and marked as such in iTop. Adjust this value depending on the scheduling recurrence. | 604800 |
 | prefix | The prefix for the name of all Synchronization Data Sources in iTop. If you run several instances of the collector (to collect information from several vSphere servers), change this value so that each data source has a unique name | vSphere | | prefix | The prefix for the name of all Synchronization Data Sources in iTop. If you run several instances of the collector (to collect information from several vSphere servers), change this value so that each data source has a unique name | vSphere |
Line 154: Line 166:
   <​brand_mapping type="​array">​   <​brand_mapping type="​array">​
     <!-- Syntax /​pattern/​replacement where:     <!-- Syntax /​pattern/​replacement where:
-      any delimiter can be used (not only /) but the delimiter cannot be present in the "​replacement"​ string+      any delimiter can be used (not only /)  
 +      ​but the delimiter cannot be present in the "​replacement"​ string
       pattern is a RegExpr pattern       pattern is a RegExpr pattern
       replacement is a sprintf string in which:       replacement is a sprintf string in which:
Line 176: Line 189:
  
 <note tip>Make sure that you keep the last (match all) item in the list to avoid loosing some values.</​note>​ <note tip>Make sure that you keep the last (match all) item in the list to avoid loosing some values.</​note>​
 +
 +
 +==== IPs and logical interfaces collection ====
 +
 +The collector automatically detects if TeemIp is present on the remote iTop application (as a module or even as a stand-alone application). Should that be the case, then the following parameters may trigger IP addresses and logical interfaces collection.
 +
 +<code xml>
 +<?xml version="​1.0"​ encoding="​UTF-8"?>​
 +  <​parameters>​
 +    ...
 +    <​teemip_options type="​hash">​
 +      <​collect_ips>​yes</​collect_ips>​
 +      <​default_ip_status>​allocated</​default_ip_status>​
 +      <​manage_ipv6>​no</​manage_ipv6>​
 +      <​manage_logical_interfaces>​yes</​manage_logical_interfaces>​
 +      </​teemip_options>​
 +    ...
 +  </​parameters>​
 +</​code>​
 +
 +^ Parameter ^ Meaning ^ Sample value ^
 +| collect_ips | Triggers IP addresses collection | yes |
 +| default_ip_status | Satus of newly created IP addresses | allocated |
 +| manage_ipv6 | Triggers IPv6 collection - not working yet | no |
 +| manage_logical_interfaces | Triggers logical interfaces collection as well as the addresses / interfaces links | yes |
 +
 +<note warning>​At this stage, IPv6 are not collected. This is due to a limitation of the collector base which cannot handle such objects yet.</​note>​
  
 ===== Usage ===== ===== Usage =====
Line 248: Line 288:
 | osversion_id | The information from ''​HostSystem -> config -> product -> fullName''​ is processed through the mapping table named ''​os_version_mapping''​| | osversion_id | The information from ''​HostSystem -> config -> product -> fullName''​ is processed through the mapping table named ''​os_version_mapping''​|
 | status | Constant value: ''​active''​ | | status | Constant value: ''​active''​ |
 +| managementip_id | Foreign key to an IPAddress object, if IP collection is activated |
  
  
Line 281: Line 322:
 | org_id | Constant value, supplied by the configuration file | | org_id | Constant value, supplied by the configuration file |
 | description | ''​VirtualMachine -> config -> annotation''​ | | description | ''​VirtualMachine -> config -> annotation''​ |
-| managementip | ''​VirtualMachine -> guest -> ipAddress''​ | 
 | cpu | ''​VirtualMachine -> config -> hardware -> numCPU''​ | | cpu | ''​VirtualMachine -> config -> hardware -> numCPU''​ |
 | ram | ''​VirtualMachine -> config -> hardware -> memoryMB''​ | | ram | ''​VirtualMachine -> config -> hardware -> memoryMB''​ |
 | virtualhost_id | either the Farm (if not empty) or ''​VirtualMachine -> runtime -> host -> name''​ | | virtualhost_id | either the Farm (if not empty) or ''​VirtualMachine -> runtime -> host -> name''​ |
 +| If TeemIp is **NOT** used ||
 +| managementip | ''​VirtualMachine -> guest -> ipAddress''​ |
 +| If TeemIp **IS** used and if IPs are collected || 
 +| managementip_id | Foreign key to an IPAddress object with ip set to ''​VirtualMachine -> guest -> ipAddress''​ |
 +
 +
 +==== IPv4 Addresses ====
 +If enabled, the information from the vSphere IPv4 addresses is imported in iTop into the IPv4Address object using the following mapping:
 +
 +^ Field in iTop ^ Source in vSphere ^
 +| org_id | Constant value, supplied by the configuration file |
 +| status | Constant value, supplied by the configuration file |
 +| short_name | ''​VirtualMachine -> guest -> hostName''​ |
 +| ip | Virtual Machine: ''​VirtualMachine -> guest -> ipAddress''​ \\ Hypervisor: ''​HostSystem → name''​ |
 +
 +==== Logical Interfaces ====
 +If enabled, the collector may register the logical interfaces attached to virtual machines. These are imported in iTop into the LogicalInterface objects using the following mapping: ​
 +
 +^ Field in iTop ^ Source in vSphere ^
 +| macaddress| ''​ VirtualMachine -> guest -> net -> macAddress ''​ |
 +| name | ''​ VirtualMachine -> config -> hardware -> device -> backing -> ''​\\ * ''​ network -> name ''​ or \\ * ''​ opaqueNetworkId ''​ or \\ * ''​ deviceName ''​ or \\ * ''​ port ''​ |
 +| virtualmachine_id | ''​VirtualMachine → name''​ |
 +| ip_list | List of IP addresses attached to the interface |
 +
 +<​note>​
 +The ip_list attribute is synchronized through the dedicated synchro data source vSphere:​lnkIPInterfaceToIPAddress.
 +</​note>​
  
 ===== Collecting information from several vSphere servers ===== ===== Collecting information from several vSphere servers =====
extensions/vsphere-data-collector.txt · Last modified: 2019/04/23 12:19 by vdumas

";