QPR ProcessAnalyzer System Architecture: Difference between revisions

From QPR ProcessAnalyzer Wiki
Jump to navigation Jump to search
No edit summary
(40 intermediate revisions by the same user not shown)
Line 1: Line 1:
QPR ProcessAnalyzer is natively a cloud-based software, and also an on-premise installation is available. Users access the QPR ProcessAnalyzer through their PCs, laptops or tables using web browser.
QPR ProcessAnalyzer is natively a cloud-based software, and also an on-premise installation is available. Users access the QPR ProcessAnalyzer through their PCs, laptops or tables using web browser.


==System Architecture ==
==System Architecture and Components==
The following figure shows the QPR ProcessAnalyzer system architecture.
The following diagram shows the overall system architecture for QPR ProcessAnalyzer with all the main components.


[[File:QPR_ProcessAnalyzer_Architecture.png|1200px]]
[[File:QPR_ProcessAnalyzer_System_Architecture.png|1150px]]


QPR ProcessAnalyer consists of the following components:
* '''QPR ProcessAnalyzer Web Client''': web-based user interface for QPR ProcessAnalyzer.
* '''QPR ProcessAnalyzer Excel Client''': Add-on to MS Excel used to admininstrate QPR ProcessAnalyzer models, datatables, scripts and users.
* '''QPR ProcessAnalyzer ScriptLauncher''': Tool to trigger QPR ProcessAnalyzer ETL scripts. QPR ProcessAnalyzer ScriptLauncher can be used for fetching on-premise data and store to the cloud.
* '''QPR UI Server''': Component in the server which handles the web UI and stores dashboards.
* '''QPR ProcessAnalyzer Server''': Component to store the eventlog data in the memory and performs analytical calculations for the data.
* '''QPR UI database''': database for the QPR UI Server.
* '''QPR ProcessAnalyzer database''': database for the QPR ProcessAnalyzer server.


== QPR ProcessAnalyzer Server API==
QPR ProcessAnalyzer consists of the following components:
All connections to QPR ProcessAnalyzer Server are done using the QPR ProcessAnalyzer API (more information: [[QPR ProcessAnalyzer Web Service API]]).
* '''QPR ProcessAnalyzer Web UI''': Web UI is a web browser based user interface for QPR ProcessAnalyzer using HTML5 and Angular as technologies. The Web UI works with all modern browsers without separate add-ons. For more information, see the [[QPR_ProcessAnalyzer_System_Requirements#System_Requirements_for_Web_Browsers|list of supported browser]].
* '''QPR ProcessAnalyzer ScriptLauncher''': ScriptLauncher is a tool to trigger QPR ProcessAnalyzer scripts used for ETL (extract, transform and load). The ScriptLauncher is installed on-premise, so it is able to fetch eventlog data from on-premise systems and store to the QPR ProcessAnalyzer cloud. In practice, ScriptLauncher can run scripts that are managed in QPR ProcessAnalyzer environment, allowing the scripts to execute data extraction commands from the on-premise source systems.
* '''QPR ProcessAnalyzer Server''': QPR ProcessAnalyzer Server is the server or cloud side component for QPR ProcessAnalyzer environment. It holds the eventlog data in-memory allowing very fast processing and calculation for the data based on the QPR ProcessAnalyzer's own expression language optimized for process mining calculations. QPR ProcessAnalyzer Server also manages users, sessions and other system configurations. QPR ProcessAnalyzer Server is based on the .Net Framework and it runs hosted by IIS (Internet Information Services). Currently Windows Server is required, but in future, migration to .Net 5 allows also support for Linux servers.
* '''QPR ProcessAnalyzer Server database''': This is the main database used by the QPR ProcessAnalyzer Server, storing for examples models, projects, dashboards, datatables, scripts, users and system configurations. Currently SQL Server is supported as the database system and connection to the database uses ''.NET Framework Data Provider for SQL Server (SqlClient)''.
* '''QPR ProcessAnalyzer scripting database''': Scripting Database is used to run SQL commands in the QPR ProcessAnalyzer scripts. The scripting database can be configured to store data only temporarily during the script runs (for the permanent storage [[QPR_ProcessAnalyzer_Project_Workspace#Datatables|datatables]] stored to the QPR ProcessAnalyzer Server Database can be used). For custom usecases, the scripting database can also have write access to store data permanently.
* '''SQL Server tempdb''': Each SQL Server instance has a database called tempdb, used to store data temporarily. The tempdb sizing and performance needs to be considered carefully when running QPR ProcessAnalyzer, as the SQL scripts use the tempdb intensively. See the [[QPR_ProcessAnalyzer_System_Requirements#Recommended_Hardware_for_QPR_ProcessAnalyzer_Database_Server|system requirements]] section for the tempdb sizing. More information about tempdb: https://docs.microsoft.com/en-us/sql/relational-databases/databases/tempdb-database?view=sql-server-ver15.
 
== Connecting to Data Sources ==
QPR ProcessAnalyzer is designed and built for easy integration to a wide range of data sources to fetch the eventlogs data. The power of process mining comes from collecting process information from differences sources to a comprehensive process mining model, where it can be analyzed from any angle. The available data sources can include for example:
* ERP systems, e.g. SAP HANA and SAP R/3 (for Order-to-Cash, Purchase-to-Pay processes etc.)
* CRM systems, e.g. Salesforce (for sales process)
* Customer support systems, e.g. Jira
* Case management systems
* Supply chain management systems
* Configuration management databases
 
To get started with an analysis, an eventlog is needed containing individual events as rows and at least following three properties as column: Case id, timestamp and event type. The eventlog can optionally contain event attributes such as sales person, location, customer, sale amount, time stamp for start and end of the event.
 
As the security of the data is essential, the architecture is built so that the data is protected when collected from the source. The data can be fetched from any source using integration interfaces whether they are located on premise or in the cloud. For example, connectors to ODBC, OLEDB, ADO.net, and SOAP/REST Web Services are available ([[SQL_Scripting_Commands|more information]]).
 
== Authentication, Authorization and Security Model ==
QPR ProcessAnalyzer supports the following methods for authenticating users:
* '''QPR ProcessAnalyzer built-in authentication''': QPR ProcessAnalyzer has its own user management which can provide password authentication for users.
* '''SAML authentication (federated authentication)''': To use organization's own identity management system (IdP), QPR ProcessAnalyzer can be integrated with it using the SAML 2.0 protocol. SAML 2.0 is also the optimal solution for QPR Cloud (more information about [[SAML_2.0_Federated_Authentication|SAML authentication]]).
* '''LDAP/AD authentication''': QPR ProcessAnalyzer can use the traditional AD/LDAP authentication, although the SAML authentication as more flexible is the recommended option (more information about [[LDAP/AD_Authentication|LDAP/AD authentication]].)


== Database Connection ==
QPR ProcessAnalyzer has its own robust and flexible authorization mechanisms that controls user's access to data. Permissions are managed for individual projects, which consist of models, datatables, dashboards and scripts. When there are lot of users, it's recommended to use ''groups'' to make user management easier: when permissions have been configured to groups, users can be easily assigned to groups. More information about [[Roles_and_Permissions_in_QPR_ProcessAnalyzer|permissions]] and [[QPR ProcessAnalyzer Session Management|user session management]].
QPR ProcessAnalyzer stores its data in an SQL Server database. Connection to the database uses .NET Framework Data Provider for SQL Server (SqlClient).


In addition to the system database, QPR ProcessAnalyzer can optionally use another database for SQL scripting for ETL (more information: [[Data_Extraction,_Transformation,_and_Loading|ETL Scripting]]).
QPR ProcessAnalyzer also supports case level permissions, allowing to determine permissions for each case separately based on flexible rules and the attribute data in the eventlog. You can for example assign permissions for cases from each company code for persons working in the respective area. More information about [[Case_Level_Permissions|case level permissions]].


== Connecting to Data Sources ==
For connection between QPR ProcessAnalyzer Server and SQL Server database, the system uses one connection string and a shared connection pool with a single SQL Server user (login) for all database operations. For easy setup, the database user can use the ''db_owner'' role, but the [[QPR_ProcessAnalyzer_Security_Hardening#Database_User_Least_Privileges|hardened configuration]] complying with the minimum privilege principle is recommended.
QPR ProcessAnalyzer is designed and built for easy integration to a wide range of data sources. The power of the product comes from having different process information accessible from one point and where it can be analyzed from any angle.


For basic analysis, Event ID, Time Stamp and Attiribute data should be available from the data source. However, depending on the customer need the analysis can be expanded by other data where all provided information can be included in the analysis be it sales person, location, customer, sale amount, time stamp for start and end of sale.
Credentials for external data sources can be managed in a separate [[Secure_Strings|secure strings storage]], which allow to store passwords and other confidential information in a secure way, that the stored credentials cannot be seen in the UI by any user.
The data sources can include:
* ERP systems e.g. SAP for Order to Cash and other processes
* CRM systems e.g. Salesforce for sales process
* Customer support systems e.g. Jira
* Case Management Systems
* Supply Chain Management systems
* Configuration Management Databases


As data security is always key, the architecture is built so that the data is protected when collected from the source. The data can be fetched from any source using integration interfaces whether they are located on premise or in the cloud.
== QPR ProcessAnalyzer Server API==
All connections to QPR ProcessAnalyzer Server are established through the [[QPR ProcessAnalyzer API]]. For example, the QPR ProcessAnalyzer Web UI and the [[QPR_ProcessAnalyzer_ScriptLauncher|QPR ProcessAnalyzer ScriptLauncher]] are based on the it, and also external integrations can use it to fetch and store data from and to QPR ProcessAnalyzer system.


[[Category: QPR ProcessAnalyzer]]
[[Category: QPR ProcessAnalyzer]]

Revision as of 13:06, 31 August 2021

QPR ProcessAnalyzer is natively a cloud-based software, and also an on-premise installation is available. Users access the QPR ProcessAnalyzer through their PCs, laptops or tables using web browser.

System Architecture and Components

The following diagram shows the overall system architecture for QPR ProcessAnalyzer with all the main components.

QPR ProcessAnalyzer System Architecture.png


QPR ProcessAnalyzer consists of the following components:

  • QPR ProcessAnalyzer Web UI: Web UI is a web browser based user interface for QPR ProcessAnalyzer using HTML5 and Angular as technologies. The Web UI works with all modern browsers without separate add-ons. For more information, see the list of supported browser.
  • QPR ProcessAnalyzer ScriptLauncher: ScriptLauncher is a tool to trigger QPR ProcessAnalyzer scripts used for ETL (extract, transform and load). The ScriptLauncher is installed on-premise, so it is able to fetch eventlog data from on-premise systems and store to the QPR ProcessAnalyzer cloud. In practice, ScriptLauncher can run scripts that are managed in QPR ProcessAnalyzer environment, allowing the scripts to execute data extraction commands from the on-premise source systems.
  • QPR ProcessAnalyzer Server: QPR ProcessAnalyzer Server is the server or cloud side component for QPR ProcessAnalyzer environment. It holds the eventlog data in-memory allowing very fast processing and calculation for the data based on the QPR ProcessAnalyzer's own expression language optimized for process mining calculations. QPR ProcessAnalyzer Server also manages users, sessions and other system configurations. QPR ProcessAnalyzer Server is based on the .Net Framework and it runs hosted by IIS (Internet Information Services). Currently Windows Server is required, but in future, migration to .Net 5 allows also support for Linux servers.
  • QPR ProcessAnalyzer Server database: This is the main database used by the QPR ProcessAnalyzer Server, storing for examples models, projects, dashboards, datatables, scripts, users and system configurations. Currently SQL Server is supported as the database system and connection to the database uses .NET Framework Data Provider for SQL Server (SqlClient).
  • QPR ProcessAnalyzer scripting database: Scripting Database is used to run SQL commands in the QPR ProcessAnalyzer scripts. The scripting database can be configured to store data only temporarily during the script runs (for the permanent storage datatables stored to the QPR ProcessAnalyzer Server Database can be used). For custom usecases, the scripting database can also have write access to store data permanently.
  • SQL Server tempdb: Each SQL Server instance has a database called tempdb, used to store data temporarily. The tempdb sizing and performance needs to be considered carefully when running QPR ProcessAnalyzer, as the SQL scripts use the tempdb intensively. See the system requirements section for the tempdb sizing. More information about tempdb: https://docs.microsoft.com/en-us/sql/relational-databases/databases/tempdb-database?view=sql-server-ver15.

Connecting to Data Sources

QPR ProcessAnalyzer is designed and built for easy integration to a wide range of data sources to fetch the eventlogs data. The power of process mining comes from collecting process information from differences sources to a comprehensive process mining model, where it can be analyzed from any angle. The available data sources can include for example:

  • ERP systems, e.g. SAP HANA and SAP R/3 (for Order-to-Cash, Purchase-to-Pay processes etc.)
  • CRM systems, e.g. Salesforce (for sales process)
  • Customer support systems, e.g. Jira
  • Case management systems
  • Supply chain management systems
  • Configuration management databases

To get started with an analysis, an eventlog is needed containing individual events as rows and at least following three properties as column: Case id, timestamp and event type. The eventlog can optionally contain event attributes such as sales person, location, customer, sale amount, time stamp for start and end of the event.

As the security of the data is essential, the architecture is built so that the data is protected when collected from the source. The data can be fetched from any source using integration interfaces whether they are located on premise or in the cloud. For example, connectors to ODBC, OLEDB, ADO.net, and SOAP/REST Web Services are available (more information).

Authentication, Authorization and Security Model

QPR ProcessAnalyzer supports the following methods for authenticating users:

  • QPR ProcessAnalyzer built-in authentication: QPR ProcessAnalyzer has its own user management which can provide password authentication for users.
  • SAML authentication (federated authentication): To use organization's own identity management system (IdP), QPR ProcessAnalyzer can be integrated with it using the SAML 2.0 protocol. SAML 2.0 is also the optimal solution for QPR Cloud (more information about SAML authentication).
  • LDAP/AD authentication: QPR ProcessAnalyzer can use the traditional AD/LDAP authentication, although the SAML authentication as more flexible is the recommended option (more information about LDAP/AD authentication.)

QPR ProcessAnalyzer has its own robust and flexible authorization mechanisms that controls user's access to data. Permissions are managed for individual projects, which consist of models, datatables, dashboards and scripts. When there are lot of users, it's recommended to use groups to make user management easier: when permissions have been configured to groups, users can be easily assigned to groups. More information about permissions and user session management.

QPR ProcessAnalyzer also supports case level permissions, allowing to determine permissions for each case separately based on flexible rules and the attribute data in the eventlog. You can for example assign permissions for cases from each company code for persons working in the respective area. More information about case level permissions.

For connection between QPR ProcessAnalyzer Server and SQL Server database, the system uses one connection string and a shared connection pool with a single SQL Server user (login) for all database operations. For easy setup, the database user can use the db_owner role, but the hardened configuration complying with the minimum privilege principle is recommended.

Credentials for external data sources can be managed in a separate secure strings storage, which allow to store passwords and other confidential information in a secure way, that the stored credentials cannot be seen in the UI by any user.

QPR ProcessAnalyzer Server API

All connections to QPR ProcessAnalyzer Server are established through the QPR ProcessAnalyzer API. For example, the QPR ProcessAnalyzer Web UI and the QPR ProcessAnalyzer ScriptLauncher are based on the it, and also external integrations can use it to fetch and store data from and to QPR ProcessAnalyzer system.