User Tools

Site Tools

:: Version 2.7.0 ::

2_7_0:install:260_to_270_migration_notes

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
2_7_0:install:260_to_270_migration_notes [2019/12/20 09:37]
vdumas [System]
2_7_0:install:260_to_270_migration_notes [2020/03/25 18:51] (current)
Line 8: Line 8:
   * For an exhaustive list of changes check the [[2_7_0:​release:​change_log|Change Log]]   * For an exhaustive list of changes check the [[2_7_0:​release:​change_log|Change Log]]
  
-====== ​To know before upgrading ​======+====== ​Impact on Users ======
  
-===== Custom extensions ​on Portal =====+  * OQL queries returning multiple classes of objects, may return less data after the upgrade, to users having "​Allowed organizations"​ set on their user. This is due to a fix in query generation.
  
-In order to ensure better security, support and sustainability;​ we migrated the portal'​s framework from Silex 2.x to Symfony 3.4. Even though we managed to keep a backward compatibility for most of the code, if you made a custom extension for the portal, you are most likely to rework part of it.+===== Portal behavior changes =====
  
-Read [[2_7_0:install:​portal_270_migration|this page]]  to check if your extension is concerned.+  * External key display in forms are now clickable, if you rather keep the previous behavior replace the //xxx_id// attribute code in your form with //​xxx_id_friendlyname//​. 
 +  * We have update Font Awesome from v4 to v5.12.0, check that your icons in portal are still displayed as expected. Some [[https://​fontawesome.com/​how-to-use/​on-the-web/​setup/​upgrading-from-version-4#​name-changes|icons names have changed]] between the 2 versions, old name will still work except if you have used aliases from the version 4, which will need to be changed. 
 +  * For security reasons, LinkedSet displayed in forms, such as for example contacts associated ​to a User Request, are now by default limited to the scope of the portal user. A typical example there, was that portal user could see on his User Requests, IT department associated contacts, even if in theory his Contact scope was limiting him to contacts of his own organization. If you want to keep the previous behavior, you need to modify the forms in your portal displaying those linksets as explained in the [[2_7_0:​customization:​portal_xml#​xml_version_17|2.7 new features in the Portal XML reference]]
  
-===== Incompatible Extensions ​=====+===== Change for Administrators users =====
  
-Because of the silex to Symphony migration, the following extensions are **not compatible with the iTop 2.7**. A new version will be published ​within the coming months ​and you will have to upgrade those extensions ​as well.+  * //Admin tools// has been modified and some of its entries moved to new //​Configuration//​ and //System// menus. 
 +  * Labels of Persons fields in User objects have been changed. This will change what you get in CSV and excel export of Users: 
 + 
 +| old label | new label| 
 +| Contact (person)<​nowiki>​-></​nowiki>​First Name ^ Person<​nowiki>​-></​nowiki>​First Name | 
 +| Contact (person)<​nowiki>​-></​nowiki>​Last Name ^ Person<​nowiki>​-></​nowiki>​Last Name | 
 +| Contact (person)<​nowiki>​-></​nowiki>​Email ^ Person<​nowiki>​-></​nowiki>​Email | 
 + 
 + 
 +====== To check before upgrading ====== 
 + 
 +===== System ===== 
 +  * Ensure that you have ''​php-gd''​ installed on your iTop server. It is now enforced at setup, as it is mandatory for handling correctly images within caselogs and description. 
 +<note important>​If you use php-memcache,​ migrate to php-memcached</​note>​ 
 + 
 +===== Setup can take time ===== 
 + 
 +The upgrade to the 2.7 product version, will generate database changes, which can take time if you have a lot of data. 
 + 
 +=== Underlying changes === 
 + 
 +__Finalclass on intermediate classes__ 
 + 
 +To increase query performance,​ we are adding on each intermediate table/class within a class hierarchy, a new column to store the final class (Only on intermediate classes as on a root class this column exist already and on a final class, this column is useless as this is the table name). 
 + 
 +As a result during Setup, for each of those intermediate classes, iTop runs an ALTER TABLE  
 +<code SQL Example>​ 
 +ALTER TABLE `priv_changeop_setatt`  
 +   ADD `optype` VARCHAR(255) CHARACTER  
 +      SET utf8mb4 COLLATE utf8mb4_unicode_ci  
 +      DEFAULT '​CMDBChangeOpSetAttribute',​  
 +   ADD INDEX `optype` (`optype` (95)) 
 +</​code>​ 
 +and just after an UPDATE table 
 +<code SQL> 
 +UPDATE `priv_changeop_setatt`,​`priv_changeop`  
 +   ​SET ​ `priv_changeop_setatt`.`optype` = `priv_changeop`.`optype`  
 +   WHERE `priv_changeop_setatt`.`id` = `priv_changeop`.`id`  
 +</​code>​ 
 + 
 +__Attachments__ 
 + 
 +To store creation date and user having uploaded an attachment, iTop Setup automatically updates the DB schema, like this:  
 +<code sql> 
 +ALTER TABLE `attachment` ADD `creation_date` DATETIME,  
 +                         ADD `user_id` INT(11) DEFAULT 0,  
 +                         ADD INDEX `user_id` (`user_id`);​ 
 +</​code>​ 
 + 
 +Warning, if you're using the table prefix option (''​db_subname''​ iTop config file parameter) the table name will vary. For example with ''​db_subname='​myprefix'''​ then the table name would be : ''​myprefix_attachment''​. 
 + 
 +=== Risks === 
 + 
 +This operation can **take a long time** if you have a **big number of attachments** and/or a **huge history** on your objects. 
 + 
 +  * ALTER TABLE execution time is roughly proportional to the table size 
 +  * UPDATE TABLE execution time is mainly impacted by the number of rows and a bit by the table size. 
 + 
 +__Examples of execution time__ 
 + 
 +Here are examples of the time it took with table size and count: 
 + 
 +| Table || Execution || 
 +| Size (GB) | count | Seconds | Query (beginning) | 
 +| 57 | 250000 | 1486 | ALTER TABLE `attachment` | 
 +| 7,3 | 32000000 | 212 | ALTER TABLE `priv_changeop_setatt` | 
 +| 7,3 | 32000000 | 2621 | UPDATE `priv_changeop_setatt`,​ | 
 +| 2,4 | 1760000 | 132 | ALTER TABLE `priv_changeop_setatt_scalar` | 
 +| 2,4 | 1760000 | 465 | UPDATE `priv_changeop_setatt_scalar`,​ | 
 +| 3,1 | 12667000 | 96 | ALTER TABLE `priv_changeop_links` | 
 +| 3,1 | 12667000 | 1076 | UPDATE `priv_changeop_links`,​ | 
 +| 1,7 | 7531000 | 69 | ALTER TABLE `priv_event_notification` | 
 +| 1,7 | 7531000 | 613 | UPDATE `priv_event_notification`,​ | 
 +| 0,8 | 143000 | 36 | ALTER TABLE `priv_changeop_setatt_longtext` ​ | 
 +| 0,8 | 143000 | 54 | UPDATE `priv_changeop_setatt_longtext`,​ | 
 + 
 +{{ :​2_7_0:​install:​longqueriesmigration27.xlsx |}} 
 + 
 +<code SQL> 
 + ALTER TABLE `priv_changeop_setatt` ADD `optype` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT '​CMDBChangeOpSetAttribute',​ ADD INDEX `optype` (`optype` (95))   
 + ALTER TABLE `priv_changeop_setatt_scalar` ADD `optype` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT '​CMDBChangeOpSetAttributeScalar',​ ADD INDEX `optype` (`optype` (95))   
 + ALTER TABLE `priv_changeop_setatt_longtext` ADD `optype` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT '​CMDBChangeOpSetAttributeLongText',​ ADD INDEX `optype` (`optype` (95))   
 + ALTER TABLE `priv_changeop_links` ADD `optype` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT '​CMDBChangeOpSetAttributeLinks',​ ADD INDEX `optype` (`optype` (95))   
 + ALTER TABLE `priv_event_notification` ADD `realclass` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT '​EventNotification',​ ADD INDEX `realclass` (`realclass` (95))   
 + ALTER TABLE `attachment` ADD `creation_date` DATETIME, ADD `user_id` INT(11) DEFAULT 0, ADD INDEX `user_id` (`user_id`) ​  
 + ​UPDATE `priv_changeop_setatt`,​`priv_changeop` SET  `priv_changeop_setatt`.`optype` = `priv_changeop`.`optype` WHERE `priv_changeop_setatt`.`id` = `priv_changeop`.`id` ​  
 + ​UPDATE `priv_changeop_setatt_scalar`,​`priv_changeop` SET  `priv_changeop_setatt_scalar`.`optype` = `priv_changeop`.`optype` WHERE `priv_changeop_setatt_scalar`.`id` = `priv_changeop`.`id` ​  
 + ​UPDATE `priv_changeop_setatt_longtext`,​`priv_changeop` SET  `priv_changeop_setatt_longtext`.`optype` = `priv_changeop`.`optype` WHERE `priv_changeop_setatt_longtext`.`id` = `priv_changeop`.`id` ​  
 + ​UPDATE `priv_changeop_links`,​`priv_changeop` SET  `priv_changeop_links`.`optype` = `priv_changeop`.`optype` WHERE `priv_changeop_links`.`id` = `priv_changeop`.`id` ​  
 + ​UPDATE `priv_event_notification`,​`priv_event` SET  `priv_event_notification`.`realclass` = `priv_event`.`realclass` WHERE `priv_event_notification`.`id` = `priv_event`.`id` ​  
 + 
 +</​code>​ 
 + 
 +=== Mitigation === 
 + 
 +To identify tables which can take time, you can use this query, modified with your true //itop db name// 
 +<code SQL> 
 +SELECT TABLE_NAME AS "​Tables",​  
 +    round(((data_length + index_length) / 1024 / 1024), 2) AS SIZE  
 +  FROM information_schema.tables  
 +  WHERE table_schema =  'itop db name'​ 
 +  ORDER BY SIZE 
 +</​code>​ 
 + 
 +<note warning>​Ensure that you have at least the size of the biggest table in free disk space on your iTop server</​note>​ 
 + 
 +Then run the upgrade process on a test environment on a copy of your production database. The ''​setup.log''​ file will provide you the exact queries which run. 
 + 
 +You can run those SQL queries on your production database **before upgrading** ! Apart from the performance degradation it can induce during the execution, there is no risk to break iTop as it will just ignore the added columns, until you upgrade it to version 2.7 
 + 
 +<note warning>​If you run manually the ALTER TABLE on a given table, **run also the UPDATE query**, otherwise your database will be incoherent and iTop 2.7 will not work. \\ There is **no** UPDATE query for ''​attachment''​ table.</​note>​ 
 + 
 + 
 +=== Setup process === 
 + 
 +<note warning>​If your browser returns a timeout during Setup database upgrade, don't kill your web server nor your database server. As despite this browser message, the Setup is going-on.</​note>​ 
 + 
 +You can connect to mysql to check the queries which are on-going, you will see the time already spent on the current query, if you run the command again, the time will increase or show the next query. 
 +<code MySQL> 
 +php>​mysql -u root -p xxxx 
 +mysql>​show processlist;​ 
 +</​code>​ 
 +As long as queries are ongoing, the setup is still going on. Then run 
 +<code PHP>tail -f setup.log 
 + 
 +2020-03-02 13:49:52 | Ok      | <---- Exiting read only mode | SetupLog 
 +</​code>​ 
 +to ensure that the setup is ended. 
 + 
 +If your browser timeout, then the Setup is not completed. Once you have ensured that the database upgrade is finished, **you must run the Setup again**, also this time it will be faster as the database upgrade will not be performed again. 
 + 
 +===== Portal ===== 
 + 
 +==== Incompatible Extensions ==== 
 + 
 +Because of the Silex to Symphony migration, the following extensions are **not compatible with iTop 2.7 Portal**.  
 +A new version will be published ​mid of April 2020 and if you have deployed them on your iTop, you must upgrade those extensions ​**before** upgrating your iTop. 
 +<note tip>If you are a **Combodo customer**, using an iTop Professional or Essential: \\ This does not apply to you!</​note>​
  
   * [[https://​store.itophub.io/​en_US/​products/​combodo-customized-request-forms|Customized request forms]]   * [[https://​store.itophub.io/​en_US/​products/​combodo-customized-request-forms|Customized request forms]]
   * [[https://​store.itophub.io/​en_US/​products/​combodo-approval-process-automation|Approval process Automation]]   * [[https://​store.itophub.io/​en_US/​products/​combodo-approval-process-automation|Approval process Automation]]
-  * [[https://​store.itophub.io/​en_US/​products/​itop-iframe-dashlet|iFrame Dashlet]]+  ​* [[https://​store.itophub.io/​en_US/​products/​combodo-approval-process-light|Approval process light]] 
 +  ​* [[https://​store.itophub.io/​en_US/​products/​itop-communications|Communications to the Customers]] 
 +  * [[https://​store.itophub.io/​en_US/​products/​customer-survey|Customer Survey]]
  
-===== End of Legacy Portal ​=====+ 
 +==== Custom extensions on Portal ==== 
 + 
 +If you have custom extensions bringing new bricks in your portal, you will need to get/write a new version of those extensions. 
 + 
 +Because, in order to ensure better security, support and sustainability;​ we migrated the portal'​s framework from Silex 2.x to Symfony 3.4. Even though we managed to keep a backward compatibility for most of the code, if you made a custom extension for the portal, you are most likely to rework part of it. 
 + 
 +Read [[2_7_0:​install:​portal_270_migration|this page]] ​ to check if your extension is concerned. 
 + 
 +==== End of Legacy Portal ====
  
 As announced, the Legacy portal is no more available in iTop 2.7 As announced, the Legacy portal is no more available in iTop 2.7
  
-====== To check before upgrading ====== 
  
-===== System ​===== +===== XML Datamodel ​=====
-  * Ensure that you have ''​php-gd''​ installed on your iTop server. It is now enforced at setup, as it is mandatory for handling correctly images within caselogs and description. +
-  * If you use php-memcache,​ migrate to php-memcached+
  
  
-===== Portal behavior changes ===== 
  
-  * In iTop 2.7, any forms which **only** propose transitions,​ are now automatically read-only. This can break customized user portalThe workaround is usually to add the ''​always_show_submit''​ tag on the form definition or to split forms to make them more specialized between creation and edition+==== Breaking changes ==== 
-  * External key display in forms are now clickable, if you rather keep the previous behavior replace ​the //xxx_id// attribute ​code in your form with //xxx_id_friendlyname//. + 
-  ​* We have update Font Awesome from v4 to v5.9.0check that your icons in portal are still displayed ​as expectedSome [[https://fontawesome.com/how-to-use/on-the-web/setup/upgrading-from-version-4#name-changes|icons names have changed]] between ​the 2 versions, old name will still work except if you have used aliases from the version 4which you will need to change+The XML default datamodel of iTop 2.7, contains a few changes ​which can collide with changes you may have done already on your iTop datamodel though extensions (//or using the ITSM Designer if your are Combodo customer//). 
-  * For security reasons, LinkedSet displayed in forms, such as for example contacts associated to a User Request, are now by default limited to the scope of the portal user. A typical example there was that portal user could see on his User Requests, IT department associated contactseven if in theory his Contact scope was limiting him to contacts of his own organization. If you want to keep the previous behavior, you need to modify the forms in your portal displaying ​those linksets as explained ​in the [[2_7_0:​customization:​portal_xml#​xml_version_17|2.7 new features in the Portal XML reference]]+ 
 +<note tip>​Those conflicts should be handled automatically during iTop Setup, if not read carefully ​the message, it explains which XML node is failing.</​note>​ 
 + 
 + 
 +=== Branding === 
 +  * The default theme is now included under ///​itop_design/​branding/​themes/​theme[@id="​light-grey"​]//​so if you altered ​the //​branding//​ node, make sure to put the alteration flag (eg. //_delta="​define"​//) on the subnodes. 
 + 
 +<code XML itop_design iTop 2.7.0> 
 +  <​branding>​ 
 +    <​themes>​ 
 +      <theme id="​light-grey">​ 
 +        ... 
 +</​code>​ 
 +__No more working:​__ 
 +<code XML itop-design ​in MyExtension>​ 
 +  <​branding _delta="​define">​ 
 +    <​main_logo>​ 
 +      <fileref ref="​logo-combodo_f737e922334a6d51b0d98e9f590d2295"​/
 +    </main_logo>​ 
 +    <​login_logo>​ 
 +      ... 
 +  </branding>​ 
 +</code> 
 +__Compatible with 2.7:__ 
 +<code XML itop-design in MyExtension>​ 
 +  ​<​branding>​ 
 +    <​main_logo _delta="​define">​ 
 +      <fileref ref="​logo-combodo_f737e922334a6d51b0d98e9f590d2295"/>​ 
 +    </​main_logo>​ 
 +    <​login_logo _delta="​define">​ 
 +      ​..
 +  </​branding>​ 
 +</​code>​ 
 + 
 +=== Legacy customers portal === 
 +The legacy customers portal was deprecated for a few releases and has now been completly removed. If you had redefined some of its constants or the routing to ityou need to correct ​your XML. 
 + 
 + 
 +__Constants:​__ Remove all constant nodes starting with "​PORTAL_" ​as they don't exist anymore. 
 +<code XML itop_design iTop 2.6 and earlier>​ 
 +<​constants>​ 
 +    <​constant id="​PORTAL_POWER_USER_PROFILE">​...</​constant>​ 
 +    <​constant id="​PORTAL_SERVICECATEGORY_QUERY">​...</​constant>​ 
 +    ... 
 +</​constants>​ 
 +</​code>​ 
 + 
 +__Routing:​__ Remove "​legacy_portal"​ from the portal nodes. 
 +<code XML itop_design iTop 2.6 and earlier>​ 
 +<​portals>​ 
 +    <portal id="​legacy_portal">​ 
 +        ... 
 +    </​portal>​ 
 +</​portals>​ 
 +</​code>​ 
 + 
 +=== Customers portal === 
 +Navigation rules have been added under ///​itop_design/​module_designs/​module_design[@id="​itop-portal"​]/​forms/​form[@id="<​FORM_ID>"​]/properties// for the "​ticket-create",​ "​ticket-edit",​ "​ticket-reopen"​ and "​ticket-apply-stimulus"​ forms, so if you altered the //​properties//​ node, make sure to put the alteration flag (eg. //​_delta="​define"//​) on the subnodes. 
 + 
 +<code XML itop_design / module_designs / module_design[id="​itop-portal"​] / forms iTop 2.7.0> 
 +  <form id="​ticket-create">​ 
 +    <​properties>​ 
 +      <​navigation_rules>​ 
 +        <​submit>​ 
 +          <​default>​go-to-open-requests<​/default>​ 
 +        </​submit>​ 
 +        ... 
 +</​code>​ 
 +__No more working:​__ 
 +<code XML itop-design / module_designs / module_design[id="​itop-portal"​] ​forms in MyExtension>​ 
 +  <form id="​ticket-create">​ 
 +    <​properties _delta="​define">​ 
 +      <​always_show_submit>​true<​/always_show_submit>​ 
 +    </​properties>​ 
 +      ... 
 +  </​form>​ 
 +</​code>​ 
 +__Compatible with 2.7:__ 
 +<code XML itop-design / module_designs / module_design[id="​itop-portal"​] / forms in MyExtension>​ 
 +  <form id="​ticket-create">​ 
 +    <​properties>​ 
 +      <​always_show_submit _delta="​define">​true</​always_show_submit>​ 
 +    </​properties>​ 
 +      ... 
 +  </​form>​ 
 +</​code>​ 
 + 
 +=== Admin Tools menus === 
 +The //Admin tools// sub-menus have been moved under the newly created group-menus //​Configuration//​ and //System//. If you have redefine those menusit may not work as expected. 
 + 
 +=== Other === 
 +  * A new //​GetTicketRefFormat()//​ method is now part of the Ticket|UserRequest|Incident|Problem|Change classesso if you added a custom method with the same name, you'll have to adjust ​your code. 
 + 
 + 
 +==== Deprecations ==== 
 +While these are not breaking changes, you should avoid using deprecated code. This is a summary of those introduced ​in the 2.7 version: 
 + 
 +  * You should verify if you make use of the deprecated ''​MetaModel::​GetNextKey'',​ such use is generally done into  /​itop_design/​classes/​class/​methods/​method[@id='​DBInsertNoReload'​] 
 +  * Portal: Action rules of type //goto// should be replaced by navigation rules. This means that instead of having an action rule in your brick, you should have a navigation rule in the corresponding form. Search for the following XPath: 
 +    * ''/​itop_design/​module_designs/​module_design/​action_rules/​action_rule/​submit''​ 
 +    * ''/​itop_design/​module_designs/​module_design/​action_rules/​action_rule/​cancel''​ 
 + 
 + 
 +====== For developers ====== 
 + 
 +===== ApplyStimulus changed ===== 
 + 
 +If the stimulus is not applicable to the object in its current state, the method ''​ApplyStimulus()''​ now returns ''​false',​ instead of ''​true''​ before
  
 +===== Removed images =====
  
 +Many images have been replaced by //Font Awesome// icons in the admin. console. Those that were not used anymore have been removed from the iTop package (///​images//​ folder). This means that if you were using them in your extensions, you should change your code accordingly.
  
 +<​code>​
 +images/​actions_bkg.png
 +images/​actions_left.png
 +images/​asc.gif
 +images/​help.png
 +images/​home.png
 +images/​menu.png
 +images/​mini-arrow-orange-open.gif
 +images/​mini-arrow-orange.gif
 +images/​mini_add.gif
 +images/​mini_search.gif
 +images/​mini_tree.gif
 +images/​newsroom-message.svg
 +images/​newsroom_menu.png
 +images/​on-off-menu.png
 +images/​onOffBtn.png
 +images/​pencil-menu.png
 +images/​printableversion.png
 +images/​reload.png
 +images/​searchBtn.png
 +images/​settings.gif
 +images/​zoom.gif
 +</​code>​
2_7_0/install/260_to_270_migration_notes.1576831065.txt.gz · Last modified: 2019/12/20 09:37 (external edit)

";