User Tools

Site Tools


extensions:ocsng-data-collector

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
extensions:ocsng-data-collector [2019/11/13 11:12]
vdumas
extensions:ocsng-data-collector [2019/11/13 11:12] (current)
vdumas
Line 1: Line 1:
-====== ​Multiple email per contact ​======+====== ​Data collector for OCS Inventory ​======
 ---- dataentry summary ---- ---- dataentry summary ----
-name                : Multiple email per contact +name                : Data collector for OCS Inventory 
-description_wiki ​   : Identify sender on multiple email addresses ​for Ticket creation from email+description_wiki ​   : Data Collector ​for OCS Inventory NG
 index_hidden ​       : yes index_hidden ​       : yes
-version ​            : 1.0.1 +version ​            : 1.0.5 
-release_dt ​         : 2019-05-16+release_dt ​         : 2019-10-29
 itop-version-min ​   :  itop-version-min ​   : 
-download_hidden ​    : http://​www.combodo.com/​itop-extensions/​ +download_hidden ​    : http://​www.combodo.com/​itop-extensions/​ocsng-data-collector-1.0.5-495.zip 
-code                : combodo-email-alias +code                : ocsng-data-collector 
-state               : ​Stable+state               : ​stable
 product_hidden ​     : not-included product_hidden ​     : not-included
-module-lists_hidden : combodo-email-alias/1.0.1 +alias-code_hidden ​  : ocsng-data-collector 
-diffusion ​          : ​+alternate-name ​     : Data collector for OCS Inventory NG 
 +module-lists_hidden : itop-data-collector-base/1.0.13, itop-data-collector-ocsng/​1.0.2 
 +diffusion ​          : ​iTop Hub,Combodo site
 ---- ----
  
  
 +<​related_components>​Other versions of this component:</​related_components>​
  
 +This //​stand-alone application//​ collects information from a **single** [[https://​www.ocsinventory-ng.org|OCS Inventory NG]] server in order to populate the iTop CMDB with the collected Servers, PCs and Virtual machines. The collector uses the iTop data synchronization engine to synchronize the information between OCS Inventory and iTop.
  
-You need to track multiple email addresses for a Contact? Then this extension is for you\\+{{ :​extensions:​itopocsngcollector.png?​direct&​450 |}}
  
-You are annoyed with [[ticket-from-email|Mail to Ticket Automation]] which does only support a single email address per Person? +An additional (and optional) extension ​[[extensions:​itop-ocsng]] is available to display the content of the OCS Inventory pages directly inside iTop, for each synchronized object (Server, PC or Virtual Machine):
-This extension will also address this issue.+
  
 +{{ :​extensions:​ocsngitopframe.png?​direct&​650 |}}
  
  
 ===== Features ===== ===== Features =====
  
-This extension ​adds a new class EmailAlias linked ​to a Contactto store alternate email addresses ​for the same Contact\\+  * Automated inventory of Brand, Model, OS Family, OS Version, Physical Server, PC and Virtual Machine from OCS Inventory server 
 +  * Automated inventory of Network Interfaces 
 +  * The collector can reside on any system with web access to iTop and mysql access to the OCS Inventory server 
 +  * Automatic creation and update of the Synchronization Data Sources in iTop. 
 +  * An optional small extension ​is available ​to display an extra tab "OCS Inventory"​ on synchronized PCsServers and VirtualMachines. This tab displays the OCS page for the device (using an IFRAME).
  
  
-When requester send an email with an email address which is not the one registered on their Person in iTop, their email request is usually rejected, because of spams. 
-With this extension, once you have defined the various email addresses associated with your Persons, the creation of ticket from email, will retrieve the right caller whatever registered address the person used. 
  
 +<note tip>This collector makes use of iTop's built-in Data Synchronization mechanism. For more information about how the data synchronization works, refer to [[latest:​advancedtopics:​data_synchro_overview|Data Synchronization Overview]] and relies on [[extensions:​itop-data-collector-base|Data collector Base]] mechanism</​note>​
  
 ===== Revision History ===== ===== Revision History =====
 ^ Date ^ Version ^ Description ^ ^ Date ^ Version ^ Description ^
-|  2019-05-16  ​| ​ 1.0.1  | Add recognition ​of related contacts ​(CCalso by their email alias +|  2019-10-29  ​| ​ 1.0.5  | Handles TeemIp :\\ - Automatically detects if TeemIp (as an iTop module or as a standalone application) is present.\\ - Optional synchronization of IPv4 addresses\\ - Optional synchronisation of IP interfaces | 
-|  ​2019-02-15  ​| ​ 1.0.0  | First version |+|  2018-09-04 ​ |  ​1.0.4  ​| ​First public version. No longer requires an alteration ​of the data model. | 
 +|  2018-06-26 ​ |  1.0.3  | New debug trace | 
 +|  2015-05-20 ​ |   ​1.0.1 ​ | A few bug fixes: support of Windows file names (i.e. c:\Program Files\...). Conversion of SPEED from '100 Mb/s' to '​100'​. Forcing a NON case-sensitive comparison for '​Laptop'​ et al. 
 +|  ​2015-02-16  ​| ​ 1.0.0  | First version | 
  
 ===== Limitations ===== ===== Limitations =====
  
-Only works with iTop 2.3.0 and above+  * The collector does not retrieve installed software information 
 +  * The collector is not normalizing Model, OS Family and OS Version 
 +  * The collector synchronizes only PCs and Servers which have a valid serial number in OCS 
 +  * If IP collection is enabled, only IPv4 addresses are collected, not IPv6 
 +  * If your iTop is accessed using HTTPS, then the same must be true for OCS Inventory otherwise the inline frame of OCS Inventory inside iTop will not display because of browsers'​s security restrictions. 
 +  * There is no single sign-on between iTop and OCS, so you'll have to login to OCS when displaying the OCS tab.
  
 ===== Requirements ===== ===== Requirements =====
  
-This extension expends [[ticket-from-email|Mail to Ticket Automation]] from version ​3.0.1, but can be used without it.\\ +  * PHP Version 5.3.0 
-It is not compatible ​with older version before [[ticket-from-email_2_6|2.6.12]] included.+  * An access to the OCS Inventory NG mysql database 
 +  * An access to the iTop web services (REST + synchro_import.php and synchro_exec.php). **Note**: starting ​with iTop 2.5.0, the user must have the profile ''​REST Services User''​ in iTop.
  
 ===== Installation ===== ===== Installation =====
 +==== Installation of the OCS Inventory integration extension ====
  
-Use the [[extensions:installation|Standard installation process]] for this extension.+=== Automated installation via iTop Hub === 
 +  * Go to the [[https://​store.itophub.io/​en_US/​products/​itop-ocsng|extensions store]] on iTop Hub 
 +  * Click on the cart icon to make the acquisition of the extension and follow the on-screen instructions to deploy it on your iTop instance 
 +=== Manual installation === 
 +  * unzip the file "​itop-ocsng.zip"​ into the iTop ''​extensions''​ folder 
 +  * make sure that the web server process has enough rights to read the copied files 
 +  * Launch iTop setup 
 +  * Select the OCS Inventory Integration extension, when prompted 
 + 
 +{{ :​extensions:​ocsngextensionsetup.png?​direct&​450 |}} 
 + 
 +  * Edit the iTop configuration file to specify the URL to access the OCS web server: 
 + 
 +<​code>​ 
 + '​itop-ocsng'​ => array ( 
 + '​ocsng_url'​ => '​http://​localhost/​ocsreports/',​ 
 + ), 
 +</​code>​ 
 + 
 +==== Install the Data collector ​for OCS Inventory ==== 
 + 
 +  * Expand the content of the zip archive "​ocsng-data-collector"​ in a folder on the machine that will run the collector application. This machine //must// have an SQL access to the OCS NG database and a web access to the iTop server. 
 +  * create the file ''​conf/​params.local.xml''​ to suit your installation,​ supplying the appropriate credentials to connect to OCS NG and iTop. 
 + 
 +By default ​this file should contains the values used to connect to iTop server and OCS NG server: 
 + 
 +<code xml> 
 +<​parameters>​ 
 +  <​itop_url>​http://​localhost/</​itop_url>​ 
 +  <​itop_login>​admin</​itop_login>​ 
 +  <​itop_password>​admin</​itop_password>​ 
 +  <​contact_to_notify>​john.doe@demo.com</​contact_to_notify>​ 
 +  <​synchro_user>​admin</​synchro_user>​ 
 +  <​sql_host>​localhost</​sql_host>​ 
 +  <​sql_database>​ocsweb;​charset=UTF8</​sql_database>​ 
 +  <​sql_login>​root</​sql_login>​ 
 +  <​sql_password>​root</​sql_password>​ 
 +</​parameters>​ 
 +</​code>​ 
 + 
 +^ Parameter ^ Meaning ^ Sample value ^ 
 +| itop_url | URL to the iTop Application | <​nowiki>​https://​localhost/</​nowiki>​ | 
 +| itop_login | Login (user account) for connecting to iTop. Must have admin rights for executing the data synchro. | admin | 
 +| itop_password | Password for the iTop account. | | 
 +| contact_to_notify | The email address of an existing contact in iTop, to be notified of the results of the synchronization |john.doe@demo.com | 
 +| synchro_user | iTop user used for synchronization web service | admin | 
 +| sql_host | The address to connect to OCS server database | default:​localhost | 
 +| sql_database | database to connect to. You can use ;charset to force the character set | default ocsweb| 
 +| sql_login | Login to connect to the OCS database | | 
 +| sql_password | Password to connect to the OCS database | |
  
 ===== Configuration ===== ===== Configuration =====
  
-No configuration ​required+By default the data collection ​configuration ​is defined in the file ''​collectors/​params.distrib.xml''​. Do not modify this file! If you need to adapt the configuration,​ create a file named ''​params.local.xml''​ in the ''​conf''​ directory and copy/paste the needed definitions into it (the structure of both XMl files is the same).  
 +This configuration defines which SQL queries have to be executed on the OCS NG server to retrieve the data and synchronize with iTop.
  
-===== Usage ===== 
  
-=== Documenting Email Aliases === +<code xml> 
-  ​- Open a Contactfor which you want to document additional email addresses +  <​default_org_id>​Demo</​default_org_id>​ 
-  ​- Go to the new tab ''​Email aliases''​ +  <​default_status>​production</​default_status>​ 
-  - Create a new Email Alias +  <​PCCollection>​yes</​PCCollection>​ 
-  - Fill the form and Submit+  <​ServerCollection>​yes</​ServerCollection>​ 
 +  <​VMCollection>​yes</​VMCollection>​ 
 +  <​OCSBrandCollector_query>​SELECT DISTINCT SMANUFACTURER as primary_key,​ SMANUFACTURER as name FROM bios</​OCSBrandCollector_query>​ 
 +  <​OCSOSFamilyCollector_query>​SELECT DISTINCT OSNAME as primary_key,​OSNAME as name FROM hardware</​OCSOSFamilyCollector_query>​ 
 +  <​OCSOSVersionCollector_query>​SELECT DISTINCT CONCAT(OSNAME,​OSVERSION) as primary_key,​OSNAME as osfamily_id,​OSVERSION as name FROM hardware</​OCSOSVersionCollector_query>​ 
 +  <​OCSServerModelCollector_query>​SELECT DISTINCT CONCAT(SMANUFACTURER,​SMODEL) AS primary_key,​SMANUFACTURER as brand_id,​SMODEL as name, '​Server'​ As type FROM bios WHERE TYPE COLLATE utf8_general_ci NOT IN ('​Notebook','​Laptop'​) AND SMANUFACTURER COLLATE utf8_general_ci NOT LIKE '​VMware%'</​OCSServerModelCollector_query>​ 
 +  <​OCSServerCollector_query>​SELECT b.SSN as primary_key,​h.ID as ocsid, h.NAME as name, h.OSNAME as osfamily_id,​h.OSVERSION as osversion_id,​ h.PROCESSORT as cpu, h.MEMORY as ram, h.IPADDR as managementip,​ b.SMANUFACTURER as brand_id,​b.SMODEL as model_id,​b.SSN as serialnumber,​ '​$default_status$'​ as status, '​$default_org_id$'​ as org_id FROM hardware AS h JOIN bios AS b ON h.id=b.hardware_id WHERE b.TYPE COLLATE utf8_general_ci NOT IN ('​Notebook','​Laptop'​) AND SMANUFACTURER COLLATE utf8_general_ci NOT LIKE '​VMware%'</​OCSServerCollector_query>​ 
 +  <​OCSServerPhysicalInterfaceCollector_query>​SELECT ​ n.ID as primary_key,​n.DESCRIPTION as name, IF(SPEED REGEXP '​^[0-9]+ ', LEFT(SPEED, LOCATE('​ ', SPEED)), SPEED) as speed, MACADDR as macaddress, IPADDRESS as ipaddress, IPMASK as ipmask,​IPGATEWAY as ipgateway,​h.NAME as connectableci_id FROM networks AS n JOIN hardware AS h ON n.hardware_id=h.id JOIN bios AS b ON h.id=b.hardware_id WHERE SMANUFACTURER COLLATE utf8_general_ci NOT LIKE '​VMware%'​ AND b.TYPE COLLATE utf8_general_ci NOT IN ('​Notebook','​Laptop'​)</​OCSServerPhysicalInterfaceCollector_query>​ 
 +  ​<​OCSPCModelCollector_query>​SELECT DISTINCT CONCAT(SMANUFACTURER,SMODEL) AS primary_key,​SMANUFACTURER as brand_id,​SMODEL as name, '​PC'​ As type FROM bios WHERE TYPE COLLATE utf8_general_ci IN ('​Notebook','​Laptop'​) AND SMANUFACTURER NOT LIKE '​VMware%'</​OCSPCModelCollector_query>​ 
 +  ​<​OCSPCCollector_query>​SELECT b.SSN as primary_key,​h.ID as ocsid, h.NAME as name, h.OSNAME as osfamily_id,​h.OSVERSION as osversion_id,​ h.PROCESSORT as cpu, h.MEMORY as ram, b.SMANUFACTURER as brand_id,​b.SMODEL as model_id,​b.SSN as serialnumber, ​'$default_status$' ​as status, '​$default_org_id$'​ as org_id FROM hardware AS h JOIN bios AS b ON h.id=b.hardware_id WHERE b.TYPE COLLATE utf8_general_ci IN ('​Notebook','​Laptop'​) AND SMANUFACTURER COLLATE utf8_general_ci NOT LIKE 'VMware%'</​OCSPCCollector_query> ​ 
 +  ​<​OCSPCPhysicalInterfaceCollector_query>​SELECT ​ n.ID as primary_key,​n.DESCRIPTION as name, IF(SPEED REGEXP '^[0-9]+ ', LEFT(SPEED, LOCATE('​ ', SPEED)), SPEED) as speed, MACADDR as macaddress, IPADDRESS as ipaddress, IPMASK as ipmask,​IPGATEWAY as ipgateway,​h.NAME as connectableci_id FROM networks AS n JOIN hardware AS h ON n.hardware_id=h.id JOIN bios AS b ON h.id=b.hardware_id WHERE SMANUFACTURER COLLATE utf8_general_ci NOT LIKE '​VMware%'​ AND b.TYPE COLLATE utf8_general_ci IN ('​Notebook','​Laptop'​)</​OCSPCPhysicalInterfaceCollector_query>​ 
 +  ​<​OCSVirtualMachineCollector_query>​SELECT h.ID as primary_key,​h.ID as ocsid, h.NAME as name, h.OSNAME as osfamily_id,​h.OSVERSION as osversion_id,​ h.PROCESSORT as cpu, h.MEMORY as ram,​h.IPADDR as managementip,​ '​$default_status$'​ as status, '​$default_org_id$'​ as org_id FROM hardware AS h JOIN bios AS b ON h.id=b.hardware_id WHERE SMANUFACTURER COLLATE utf8_general_ci LIKE '​VMware%'</​OCSVirtualMachineCollector_query>​ 
 +  <​OCSLogicalInterfaceCollector_query>​SELECT ​ n.ID as primary_key,​n.DESCRIPTION as name, IF(SPEED REGEXP '^[0-9]+ ', LEFT(SPEED, LOCATE('​ ', SPEED)), SPEED) as speed, MACADDR as macaddress, IPADDRESS as ipaddress, IPMASK as ipmask,​IPGATEWAY as ipgateway,​h.NAME as virtualmachine_id FROM networks AS n JOIN hardware AS h ON n.hardware_id=h.id JOIN bios AS b ON h.id=b.hardware_id WHERE SMANUFACTURER COLLATE utf8_general_ci LIKE '​VMware%'</​OCSLogicalInterfaceCollector_query>​ 
 +</​code>​ 
 +==== Principles ====
  
-{{ :​extensions:​email-alias-create.png?nolink |}}+  * For each class of object to import into iTop (PC, Server, VM, Brand, Model, OSVersion OSFamily), a Synchro Data Source is created by the collector application. 
 +  * Each collector is associated with an SQL query (via the parameter ''<​nowiki><​name_of_the_collector>​_query</​nowiki>''​). 
 +  * This query must return column where aliases (i.e. "​name"​ of the columns) correspond to the expected fields for the corresponding Synchro Data Source. 
 +  * These queries can be adapted to suit your needs (for example to better differentiate PCs and Servers or to change/set some default values for some columns) 
 +  * Since the synchronization of PCs, Servers and Virtual Machines all rely on their own SQL query, it is important to make sure that the results of these 3 queries do no overlap. Otherwise the same "​system"​ will be imported several times in itop (for example as both a PC and a Server).
  
-You can create multiple Email Aliases ​for the same Person +^ Parameter ^ Meaning ^ 
-{{ :​extensions:​email-alias-edit.png?​nolink ​|}}+|default_org_id|Define the default organization ​for the synchronized CIs| 
 +|default_status|Define the default status for the synchronized CIs| 
 +|PCCollection|Define wether PC collection is active (yes) or not (no)| 
 +|ServerCollection|Define wether Server collection is active (yes) or not (no)| 
 +|VMCollection|Define wether Virtual machine collection is active (yes) or not (no)| 
 +|OCSBrandCollector_query| SQL query to retrieve from OCS NG server the list of known Brands| 
 +|OCSOSFamilyCollector_query| SQL query to retrieve from OCS NG server the list of known OS Families| 
 +|OCSOSVersionCollector_query| SQL query to retrieve from OCS NG server the list of known OS Version| 
 +|OCSServerModelCollector_query| SQL query to retrieve from OCS NG server the list of known Physical server models| 
 +|OCSServerCollector_query| SQL query to retrieve from OCS NG server the list of physical servers| 
 +|OCSServerPhysicalInterfaceCollector_query| SQL query to retrieve from OCS NG server the list of physical network interfaces for servers| 
 +|OCSPCModelCollector_query| SQL query to retrieve from OCS NG server the list of known PC models| 
 +|OCSPCCollector_query| SQL query to retrieve from OCS NG server the list of PCs| 
 +|OCSPCPhysicalInterfaceCollector_query| SQL query to retrieve from OCS NG server the list of physical interfaces for PCs| 
 +|OCSVirtualMachineCollector_query| SQL query to retrieve from OCS NG server the list of virtual machines 
 +|OCSLogicalInterfaceCollector_query| SQL query to retrieve from OCS NG server the list of logical network interfaces for virtual machines|
  
-| Field | Usage | +These queries can be redefined ​in the file conf/​params.local.xml in order to take into account your specific needs (For instance change de status of created Server, PC, Virtual Machine, as well as de default organisation). By default, only VMWare virtual machines are created. The Virtualmachine query has to be adapted if you want to handle other type of VMs
-| Email | the email address ​in unix format | +
-| Contact | the Contact associated with this email address | +
-| Type | to differentiate:​\\ ​ ''​Personal'':​ non-professional email\\ ​ ''​Professional - by function'':​ associated with a **function**\\ ​ ''​Professional - by name'':​ based on Person **name** | +
-| Comment | Free text |+
  
-=== Moving a functional email to another Person === 
-Later a functional email address may not be linked to the right person anymore, in which case you can edit it and change the associated Contact. 
-{{ :​extensions:​email-alias-modif.png?​nolink&​400 |}} 
  
 +<note tip>The file ''​params.distrib.xml''​ contains the default values for the parameters. Both files (''​params.distrib.xml''​ and ''​params.local.xml''​) use exactly the same format. But ''​params.distrib.xml''​ is considered as the reference and should remain unmodified. Should you need to change the value of a parameter, copy and modify its definition in ''​params.local.xml''​. The values in ''​params.local.xml''​ have precedence over the ones in ''​params.distrib.xml''</​note>​
  
-=== Notifying on Email Alias ===+==== IPs and IP Interfaces collection ====
  
-It is possible to notify ​the persons using one of their email aliasbased on its type for example+The collector automatically detects if TeemIp ​is present on the remote iTop application (as a module or even as a stand-alone application) and if TeemIp Zone Management extension is installed. Should that be the casethen the following parameters may trigger IP addresses and IP interfaces collection. 
-<​code ​SQL+ 
-SELECT EmailAlias AS e JOIN Person AS p ON e.contact_id ​p.id JOIN [...] WHERE e.type = "​functional"​+<​code ​xml
 +<?xml version="​1.0" encoding="​UTF-8"?>​ 
 +  <​parameters>​ 
 +    ​... 
 +    <!-- TeemIp options --> 
 +    <​collect_ips>​yes</​collect_ips>​ 
 +    <​default_ip_status>​allocated</​default_ip_status>​ 
 +    <​manage_ipv6>​no</​manage_ipv6>​ 
 +    <​default_view_name></​default_view_name>​ 
 +    ​..
 +  </​parameters>​
 </​code>​ </​code>​
  
 +^ Parameter ^ Meaning ^ Sample value ^
 +| collect_ips | Triggers IP addresses collection | yes |
 +| default_ip_status | Satus of newly created IP addresses | allocated |
 +| manage_ipv6 | Triggers IPv6 collection - not working yet | no |
 +| default_view_name| View of the newly created IP addresses | <empty string> |
 +
 +<note warning>​At this stage, IPv6 are not collected. This is due to a limitation of the collector base which cannot handle such objects yet.</​note>​
 +
 +
 +===== Usage =====
 +
 +To launch the data collection and synchronization with iTop, run the following command (from the root directory where the data collector application is installed):
 +
 +<​code>​
 +php exec.php
 +</​code>​
 +
 +The following (optional) command line options are available:
 +
 +^ Option ^ Meaning ^ default value ^
 +| <​nowiki>​--console_log_level=<​level></​nowiki>​ | Level of output to the console. From -1 (none) to 9 (debug). | 6 (info) |
 +| <​nowiki>​--collect_only</​nowiki>​ | Run only the data collection, but do not synchronize the data with iTop | false |
 +| <​nowiki>​--synchro_only</​nowiki>​ | Synchronizes the data previously collected (stored in the ''​data''​ directory) with iTop. Do not run the collection. | false |
 +| <​nowiki>​--configure_only</​nowiki>​ | Check (and update if necessary) the synchronization data sources in iTop and exit. Do NOT run the collection or the synchronization |
 +| <​nowiki>​--max_chunk_size=<​size></​nowiki>​ | Maximum number of items to process in one pass, for preserving the memory of the system. If there are more items to process, the application will iterate. | 1000 |
 +
 +The execution of the command line will:
 +  - Connect to iTop to create the Synchronization Data Sources (or check their definition if they already exist, updating them if needed)
 +  - Connect to OCS NG to collect the information about the Servers, PC, Virtual Machines
 +  - Upload the collected data into iTop (this is the configuration parameter ''<​name_of_the_collector>​_query''​
 +  - Synchronize the collected data with the existing iTop CIs.
 +
 +==== Scheduling ====
 +
 +Once you've run the data collector interactively,​ the next step is to schedule its execution so that the collection and import occurs automatically at regular intervals.
 +
 +The data collector does not provide any specific scheduling mechanism, but the simple command line ''​php exec.php''​ can be scheduled with either [[http://​en.wikipedia.org/​wiki/​Cron|cron]] (on Linux systems) or  using the [[http://​en.wikipedia.org/​wiki/​Windows_Task_Scheduler|Task Scheduler]] on Windows.
 +
 +<note tip>For optimal results, don't forget to adjust the configuration parameter ''​full_load_interval''​ to make it consistent with the frequency of the scheduling.</​note>​
  
extensions/ocsng-data-collector.txt · Last modified: 2019/11/13 11:12 by vdumas

";