Due to the multifaceted potential business opportunities that artificial intelligence offers, numerous organizations are currently developing initiatives for information-driven innovations. They establish analytics teams, create new positions for algorithmic specialists, build internal expertise, and demand from IT organizations an infrastructure for intensive computational engineering and processing, along with the provision of an analytical toolkit. For IT architects, fascinating challenges await, including collaboration with interdisciplinary teams whose members possess varying degrees of knowledge in artificial intelligence and different requirements for tool support.
Several considerations emerge: Should algorithmic specialists work with computational toolkits and develop custom tailored algorithms only in exceptional cases, thereby avoiding later challenges through unconventional integrations? Do algorithmic functions make sense in proven ETL tools or within databases? Should ambitious business users eventually prepare and link raw information themselves, apply popular algorithms to prepared datasets, and interpret results independently?
For these questions, emerging and established software manufacturers, along with the open source community, offer comprehensive solutions or computational learning extensions. Against the backdrop of the analytical process, which describes the journey of an algorithmic model from experimental phase to operationalization, this article compares selected approaches, examining notebooks for information analysis, computational learning components in ETL and visualization tools versus specialized solutions for artificial intelligence, and considers possible application areas and integration aspects.
Analytical Process and Teams
During the big information revolution, design patterns for large-scale analytical architectures emerged, along with terminology definitions that distinguish disciplines like information integration from computational engineering and algorithmic analysis. Process models, such as CRISP-DM (Cross-Industry Standard Process for Information Mining) developed from 1996 within an EU funding project, and best practices for organizing successfully operating analytical teams point the direction for how organizations can extract maximum value from their information treasures.
The discipline of algorithmic analysis describes the scientifically-oriented process of utilizing internal and external information sources to optimize products, services, and processes through application of statistical and mathematical models. The following diagram presents individual phases of the analytical process against participating functions, summarizing practical experiences. The intensity of collaboration between algorithmic specialists and system engineers is particularly high during preparation and provision of required information sources and later during productive deployment of results.
Intensive utilization of server infrastructure occurs in all phases where hands-on work with the information pool happens, often massively parallel, such as during information preparation, training of algorithmic models, and similar tasks.
Technology giant Google employees examined real computational learning systems and discovered that implementation effort for the actual core (the algorithmic code, represented by the small black box in the center) is minimal when compared to providing comprehensive and complex infrastructure including management functions.
Conceptual Architecture for Artificial Intelligence and Analytics
Utilizing all available information for analysis, conducting analytical projects, with resulting measures for process optimization and automation, means organizations must face new challenges: introducing new technologies, applying complex mathematical methods, and new working approaches that have not existed in this form previously. For IT architects, there is abundant work, either to build a new information management platform or to further develop existing information management.
A four-tier architecture oriented toward analytics and computational learning addresses these needs. What has changed compared to traditional information warehouse and business intelligence architectures from the 1990s? Consider precision manufacturing of complex products aiming to further reduce waste and achieve higher productivity increases in production lines: Manufacturing modules involved in product creation measure and detect process states through numerous sensors, programmable logic controllers regulate procedures, and capture high-resolution photographs of end products for control purposes.
In this scenario, substantial interesting measurement information emerges, often already utilized in operational activities, such as real-time alerting when thresholds are exceeded or not met within predefined process windows. While previously perhaps only status information and fault data found their way into relational databases for cost reasons, today raw information like time series (force effects, feed rates, voltage, frequencies) are preserved for later analysis.
Regarding the acquisition area, IT architecture now handles tasks like capturing and storing machine and sensor information that generates data points in millisecond intervals. While IoT platforms facilitate registering, connecting, and managing hundreds or thousands of such information-producing devices, the associated IT concept describes handling protocols like MQTT, OPC-UA, building and deploying messaging platforms for publish-subscribe models to efficiently process mass information in JSON format.
In the organization area, distributed NoSQL databases are increasingly established alongside relational databases for persisting incoming information streams, as they emerge in the described production scenario. For high-resolution images, audio, video recordings, or other unstructured information, object storage serves as an alternative storage form. Beyond cost-effective and long-lasting information preservation, the ability to flexibly describe individual objects with metadata enables later discoverability and provides necessary context for analyses.
With the right technology mix and consistent implementation of an information lake or virtual warehouse concept, IT architects succeed in supporting diverse analytical use cases.
Feature Engineering and Model Development
Within the analytical process, besides secure and massive information storage and the ability for simultaneous, parallel processing of large information volumes, feature engineering plays an important role. Returning to the manufacturing example: computational learning should identify unknown reasons for excessive waste. What are the determining factors? Does machine configuration influence something, or do frequency changes in a wear component over time indicate problems?
Machines and sensors deliver many parameters as time series information, but only some are relevant for the task at hand, possibly only in specific combinations. Therefore, algorithmic specialists attempt to improve prediction or classification performance of learning algorithms during feature development by creating characteristics from raw information and simplifying the learning process with these features.
Subsequent feature selection attempts to reduce the dimensionality of the training problem by choosing the most important subset of original information features. Due to these and other work steps, such as selecting and training suitable algorithms, building a computational learning model is an iterative process where specialists construct dozens or hundreds of models until acceptance criteria for model quality are fulfilled.
From a technical perspective, IT architecture should optimally support computational learning model management, including model versioning, deployment, tracking in production environments, and automating retraining processes.
Analysis and Delivery Capabilities
The analysis and delivery areas show familiar analytical capabilities, such as providing standard reporting, self-service functions for business planning, and ad-hoc analysis and exploration of new datasets. Analytical activities can substantively complement established business intelligence platforms by making existing reporting smarter through novel metrics and possibly contributing predictions for insights into the near future.
Computational learning services or products are alternative delivery forms for optimizing business processes through analytics. For example, integrated into call center applications that display churn indicators for currently calling upset customers along with scores for their departure willingness and recommended actions (vouchers, discounts). Customer scores or other risk assessments are provided by service interfaces that can be integrated and queried in real-time by various internal or external applications.
Work areas for IT architecture in this context include providing and operating scalable algorithmic models via REST APIs in production environments, including protection against unwanted access.
Classical Approach: Information Analysis with Jupyter Notebooks and Python
Jupyter is a command-line interpreter for interactive work with the Python programming language. It represents not merely an extension of Python’s built-in shell, but a software suite for developing and executing Python programs. Functions like introspection, command-line completion, rich media embedding, and various editors (terminal, Qt-based, or browser-based) enable comfortable development and simultaneous documentation of Python applications and computational learning projects.
Information analysts working with Jupyter are not limited to Python as a programming language but can also integrate kernels for Julia, R, and many other languages. A Jupyter notebook consists of a series of cells arranged in sequence. Each cell can contain either text or live code and is arbitrarily movable. Texts can be formatted in cells using simple markup language, and complex formulas can be displayed with LaTeX-like output.
Code cells contain code in the programming language assigned to the active notebook through the corresponding kernel (Python 2, Python 3, R, etc.). Notebooks allow entire computational learning projects from information acquisition to model evaluation to be reproducibly mapped and are easily versioned. Complex algorithmic models can be made transportable in Python using the Pickle module, which implements serialization and deserialization algorithms.
A problem quickly encountered in practical work with local Jupyter installations can be described as the “works on my machine” syndrome. Small datasets function problem-free on local computers, but when migrated to production dataset sizes, reading and processing all information with a single computer doesn’t scale. Due to this limitation, building a server-based computational learning environment with sufficient computational and storage capacity is obvious.
However, setting up such an environment, particularly with on-premise infrastructure, is challenging: infrastructure teams must provide and integrate physical servers and virtual machines on demand. This approach is time-consuming and error-prone due to many manual work steps. Using cloud-based technologies significantly simplifies this process.
The ability to use infrastructure on demand and combine it with scalable cloud warehouses offers immediate access to computational and storage resources whenever needed and reduces administrative effort in setting up and managing deployed computational learning software.
Cloud-Based Scaling Solutions
Advanced implementations show code excerpts from Jupyter notebooks provided within cloud services that access multi-node Apache Spark clusters through PySpark kernels. In such scenarios, larger datasets with hundreds of millions of records are completely loaded from cloud warehouses via Spark connectors into Spark dataframes and further processed in Spark clusters.
With Spark clusters and modern cloud warehouses, two powerful environments for computationally intensive tasks are available independently. With current connectors, intelligent work distribution through query pushdown is achievable, where Spark’s optimizer decides which tasks are delegated to the warehouse due to more efficient processing.
Selecting Appropriate Computational Learning Frameworks
Before examining additional tools and technologies, it makes sense for both algorithmic specialists and IT architects to first gain an overview of currently available computational learning frameworks. From an architectural perspective, understanding which tasks respective frameworks address, what technical requirements exist, and possible dependencies on available information sources is important.
A common denominator of many failed computational learning projects is frequently selecting the wrong framework. For example, TensorFlow is currently one of the most important frameworks for programming neural networks, deep learning models, and other algorithmic learning algorithms. While deep learning perfectly suits examining complex information like image and audio files, it’s increasingly used for use cases where other frameworks are better suited.
A compact decision matrix for currently widespread frameworks addresses frequent practical problems: either algorithms unsuitable or barely suitable for the use case are used, or the chosen framework cannot handle arising information volumes. The division of frameworks into small, big, and complex information categories is somewhat simplified but should help in selecting frameworks based on information type and volume.
The boundary between big and small information is drawn where information volumes become so large that they can no longer be evaluated on a single computer but must be processed in a distributed cluster. Complex information in this matrix represents unstructured information like image and audio files, for which deep learning frameworks are very well suited.
Self-Service Computational Learning in Business Intelligence Tools
With easy-to-use business intelligence tools for information visualization, analysts and less technically versed users can quite easily present complex information meaningfully in interactive dashboards. Manufacturers like Tableau, Qlik, and Oracle demonstrate their strengths particularly in visual analytics areas.
Instead of sending static reports or Excel files before the next meeting, modern meeting and creative spaces allow interactive information analyses on smart boards including changing query filters, perspective shifts, and drill-downs. Within analytical projects, these tools can be meaningfully employed both for information exploration and for visualizing results of complex computational learning models.
Predictions, scores, and other algorithmic model outputs can be understood faster and support decision-making and derivation of next measures for business processes. Within an overall IT architecture, analysis notebooks and visualization tools are essential for the standard analytics toolbox of enterprises.
Regarding efficient team collaboration, internal exchange, and communication of results, focus should not be solely on desktop tools, but server solutions should be considered and introduced together with user concepts to reduce tenfold report duplications and competing statistics.
Additional statistical functions up to the ability to execute R and Python code during analysis opens the door to computational learning for business users. Advanced visualization tools enable analysis of complex datasets with heatmap visualizations to highlight expensive residential areas. With few clicks, descriptive statistics application is possible, allowing direct reading of location measures (median, quartile values), dispersion measures (range, interquartile range), and distribution shape from visualizations.
The advantage of these visualizations is their high information density, which however must be correctly interpreted by users. When assessing attributes with their value expressions and dependencies within datasets, citizen analysts still require mathematical and statistical foundations to avoid misinterpretations.
The utility of integrated computational learning models in visualization tools is questionable: technically, results can be achieved and model quality can be evaluated and compared with other models based on some performance metrics, but who can scientifically defend the achieved results? The same applies to integration of existing R and Python scripts, which ultimately require user training regarding model parameterization and interpretation assistance for achieved results.
Computational Learning in and with Databases
Using embedded one-click analytics functions of the presented visualization tools is undoubtedly comfortable and suitable for quick experimentation. The opposite and more purist approach would be implementing custom computational learning models in databases. For implementing chosen algorithms, existing database capabilities suffice: SQL including mathematical and statistical SQL functions, tables for storing results and algorithmic model management, and stored procedures for mapping complex business logic and workflow control.
As long as algorithms are sufficiently scalable, there are many good reasons to use warehouse engines for computational learning:
Simplicity – no need to manage another compute platform, integrate between systems, and extract, transfer, load, analyze information, etc.
Security – Information remains where it’s well protected. No need to configure database credentials in external systems or worry about where information copies might be distributed.
Performance – A good warehouse engine manages many metadata for optimizing SQL queries that could also be reused during the computational learning process, an advantage over general-purpose compute platforms.
Implementation of minimal but legitimate algorithmic algorithms can be demonstrated through decision trees in cloud warehouses. Decision trees are used for building regression or classification models, where datasets are divided into increasingly smaller subsets organized in tree structures.
However, the development and implementation process for computational learning models is more comprehensive: relevant information must be identified and prepared for the algorithmic model. Simply extracting raw or non-aggregated information from database tables is insufficient; instead, an algorithmic model requires as input a flat, usually very wide table with many aggregates called features.
Only then can the process continue with training the selected algorithm for the task and evaluating model quality. If results are satisfactory, implementing the algorithmic model in the target environment follows and must prove itself when scoring fresh datasets in the future.
Many time-consuming subtasks exist where at least partial automation would be desirable. Information preparation alone can consume up to 70-80% of total project time. Implementation of algorithmic models is also frequently underestimated, as production environments have defined technology stacks that may need extension for computational learning tasks.
Therefore, it’s appealing when database management systems are also applicable here, provided required algorithms can be mapped there. Advanced tools can accelerate development of algorithmic models for customer churn prediction and deploy them in cloud warehouses: required information extracts are quickly unloaded and provide input for new projects.
Once necessary table linkages and other professional information are stored, tools analyze information structures and transform all input tables into flat intermediate tables (possibly with hundreds of columns), based on which algorithmic models are subsequently trained. After model training, results examination follows: the created dataset, model quality, and generated SQL ETL code for creating intermediate tables, as well as SQL representation of the algorithmic model that calculates probability values based on input information and stores them in scoring tables.
Advantages of this approach are obvious: shorter project times, deployment within cloud warehouses makes experimenting with dedicating computational resources for performant processing extremely simple. Limitations lie in available algorithms.
Specialized Software Suites for Computational Learning
While established business intelligence and integration tools position themselves with extensions for executing Python and R code as necessary components of the analysis toolbox for analytical processes, there are also computational learning platforms tailored to working with artificial intelligence.
For getting started in algorithmic analysis, often available open-source distributions are suitable, which also have enterprise versions with extended possibilities for accelerated computational learning through graphics processors (GPUs), better scaling, and functions for algorithmic model management (such as version management and automation).
A popular computational learning suite is the open source H2O project. The solution from the California-based company of the same name has an R interface and enables users of this statistical programming language to gain performance advantages. Functions and algorithms available in H2O are optimized and thus a good alternative to the function set already standard in R packages.
H2O implements algorithms from statistics, information mining, and computational learning (generalized linear models, K-Means, Random Forest, Gradient Boosting, and Deep Learning) and offers good foundation for training complex computational learning models faster with in-memory architecture and standard parallelization across all available processor cores.
The web-based H2O Flow interface combines the described Jupyter notebook approach with additionally integrated user guidance for the most important process steps of computational learning projects. With several clicks, datasets can be imported, converted into H2O-specific dataframes, and subsequently divided into training and test datasets.
Selection, configuration, and training of computational learning models occur either through users in beginner, advanced, or expert modes, or in auto-ML mode. Following this, H2O Flow allows prediction for target variables for still unknown datasets and preparation of result sets. What support H2O offers for productive deployment of algorithmic models will be considered through examples in following sections.
From Prototype to Productive Computational Learning Solutions
Why is it still difficult for many organizations to derive value from their first analytical activities and research labs? In practice, only through operationalizing computational learning results in production environments does real business value emerge, and only in daily business do robust algorithmic models with high quality help achieve set organizational goals.
Unfortunately, the path from prototype to productive deployment proves difficult for many initiatives. A typical scenario illustrates this: analytical teams find it technically easy in their laboratory environments to create prototypes of powerful algorithmic models using current frameworks like TensorFlow, Keras, and Word2Vec on their laptops or in sandbox environments.
However, depending on available infrastructure, due to limitations in computational power or main memory, only a subset of production information can be used for training algorithmic models. Result presentations to stakeholders of analytical projects then occur rather through storytelling in presentations or through demonstrators, rarely technically implemented so that other applications can make use of new risk scoring, image analysis modules, etc., testwise via REST APIs.
Equipped with management approval, analytical teams hand over trained algorithmic models to software engineering teams. After handover, however, engineering teams must ensure that algorithmic models are reimplemented in programming languages acceptable for production operation, such as Java, to meet IT enterprise standards and requirements for scalability and runtime behavior.
Sometimes deviations in algorithmic model output and in any case significant time losses during deployment are to be feared with such extra steps.
If analytical teams actively support deployment, then embedding newly developed algorithmic models in web applications would be a popular variant, where typically Flask, Tornado (both micro-frameworks for Python), and Shiny (an HTML5/CSS/JavaScript framework based on R) are used as technology components.
In this approach, algorithmic models, information, and used packages/dependencies must be packaged in a format that runs both in analytical sandboxes and on production servers. For large enterprises, this can mean a lengthy, complex software delivery process that may first need to be established.
In this context, the question arises how far analytical team experience extends in developing web applications and whether aspects like load balancing and network traffic are sufficiently considered. Container virtualization, such as with Docker, for isolating individual applications and elastic cloud solutions that provide required computational power on demand can provide relief and be part of the solution architecture.
Depending on analytical tasks, appropriate technical design must be chosen: Should the algorithmic model work in batch or near real-time mode? Should caching be provided for recurring model requests? How is model deployment implemented: in-memory, code-independent through exchange formats like PMML, serialized via R or Python objects (Pickle), or through generated code?
Additionally, for productive deployment of algorithmic models, supporting concepts for provision, routing, version management, and operation at industrial scale must be developed so that reliable computational learning products and services for internal and external use can emerge.
Code Generation and Deployment Strategies
The deployment variant “computational learning code generation” can be well illustrated using the previously discussed H2O Flow example. While previous sections showed steps for model building, training, and testing, the following demonstrates the download process for previously generated Java code for building algorithmic models for predicting housing prices.
In the generated Java code, information preparation performed in H2O Flow and all configurations for the Gradient Boosting Machine (GBM) algorithm are well traceable, providing initial impressions and recalling similar approaches previously described with cloud warehouses and specialized tools.
After completing computational learning development, the Java code of new algorithmic models can be added to streaming applications using Apache Kafka Streams API and published. The advantage: Kafka Streams applications are themselves Java applications into which generated algorithmic model code can be embedded.
All future events containing new real estate datasets with the same features as geographic position, building age, number of rooms, etc., that come in as algorithmic model input via Kafka Streams, are answered with predictions of probable building prices from algorithmic algorithms trained on historical information.
An advantage: Because Kafka Streams applications use all Apache Kafka functions under the hood, this new application is already designed for scalable and business-critical deployment.
Computational Learning as a Service – API-First Approach
Previous sections addressed challenges when transferring results of information experiments to production environments. While the majority of analytical team members prefer using R, Python (and increasingly Julia) as programming languages, there are software engineers on the recipient side responsible for technical implementations in production environments who possibly must use a completely different technology stack.
In extreme cases, reimplementing computational learning models threatens; in better cases, code or algorithmic model specifications can be transferred and embedded with little effort or directly made executable in new runtime environments. Alternatively, an “API-first” approach is chosen, decoupling interaction of differently implemented applications through web APIs.
Analytical teams make URL endpoints of their test-ready algorithms known, which are consumed by other software developers for their own intelligent applications. By building REST APIs, analytical teams can develop their algorithmic model code separately from other teams, thus creating work division with clear responsibilities without blocking teammates not involved in computational learning aspects of projects.
A simple scenario shows object recognition of arbitrary images implemented with deep learning procedures. Individual photos can be transmitted via command-line editor as input for image analysis to pretrained computational learning models. Information about recognized objects including probability values comes back as JSON output in return.
For implementing this example, a pretrained algorithmic model was made callable via REST API in Python based on the open source deep learning library Keras using the Flask micro web framework. The described application also handles loading arbitrary images via cURL, preprocessing them (possibly converting to RGB, standardizing image size to 224 x 224 pixels), and then passing them to the algorithmic model for classifying depicted objects.
The algorithmic model itself uses a ResNet50 architecture (abbreviation stands for 50 Layer Residual Network) and was pretrained based on the public ImageNet image database. For algorithmic model input (football players in action), the system reports comprehensible objects like football, volleyball, and jersey for testers, while questionable classifications are flashlight (torch) and wheelbarrow (barrow).
When building and providing computational learning functions through REST APIs, IT architects and involved teams consider whether the intended use is rather rapid prototyping or widespread usage must be supported. While the described scenario with Python, Keras, and Flask can be realized on a laptop, scalable deep learning solutions require more attention regarding deployment architecture, where additionally a message broker with in-memory datastore buffers incoming images to be analyzed and then forwards them for batch processing.
Using an upstream web server, load balancer, and graphics processors (GPUs) are further conceivable components for productive algorithmic architecture.
As a concluding example of a powerful (and paid) computational learning service, Google Cloud Vision image analysis should serve. Providing the same image with the football player scene, the Google algorithmic service recognizes far more information beyond objects: context (team sport, league), the player himself through facial recognition, and current and previous team affiliations, etc.
This clearly shows through the tech giant’s example: it depends primarily on available training information to what extent algorithms and appropriate automation can generate new insights without lengthy and expensive manual effort. Some organizations will discover that their own, perhaps unique, information treasure has real monetary value.
Integration Considerations and Architecture Patterns
Modern analytical architectures must accommodate diverse computational learning approaches while maintaining operational efficiency. The integration of algorithmic capabilities into existing IT landscapes requires careful consideration of several architectural patterns:
Hybrid deployment models combine on-premise computational resources with cloud-based scalability, allowing organizations to maintain sensitive information internally while leveraging external computational power for intensive training tasks. This approach provides flexibility in handling varying workloads while addressing compliance requirements.
Microservices architectures enable modular deployment of algorithmic components, where individual models can be developed, tested, and deployed independently. This pattern facilitates continuous integration and deployment practices while reducing the risk of system-wide failures due to individual model updates.
Event-driven architectures support real-time analytical processing by enabling algorithmic models to respond to information streams as they occur. This pattern is particularly valuable for applications requiring immediate responses, such as fraud detection or predictive maintenance systems.
Performance Optimization and Scaling Strategies
Effective computational learning implementations require sophisticated approaches to performance optimization. Caching strategies reduce computational overhead by storing frequently requested predictions, while intelligent load balancing distributes processing across available resources.
Resource allocation strategies must account for the variable computational demands of different algorithmic approaches. Deep learning models typically require graphics processing units for efficient training and inference, while traditional statistical models may perform adequately on standard processors.
Memory management becomes critical when processing large datasets or maintaining multiple model versions simultaneously. Efficient memory utilization strategies include streaming processing for large datasets and model pruning techniques to reduce memory footprint without significantly impacting accuracy.
Security and Governance Frameworks
Implementing computational learning capabilities introduces new security considerations that must be addressed through comprehensive governance frameworks. Model versioning and audit trails ensure reproducibility and accountability in algorithmic decision-making processes.
Access control mechanisms must be implemented to protect sensitive training information and prevent unauthorized model modifications. Role-based access controls ensure that only qualified personnel can modify production models or access sensitive algorithmic implementations.
Information privacy considerations become paramount when implementing algorithmic systems that process personal or sensitive business information. Privacy-preserving techniques such as differential privacy and federated learning enable computational learning while protecting individual privacy rights.
Monitoring and Maintenance Strategies
Productive computational learning systems require continuous monitoring to ensure optimal performance and accuracy over time. Model drift detection identifies when algorithmic models begin to lose accuracy due to changing information patterns, triggering retraining processes.
Performance monitoring encompasses both technical metrics, such as response times and computational resource utilization, as well as business metrics, such as prediction accuracy and model effectiveness in achieving business objectives.
Automated maintenance procedures enable systems to adapt to changing conditions without manual intervention. This includes automated retraining schedules, model validation processes, and rollback mechanisms for problematic deployments.
Final Thoughts
The computational learning landscape continues evolving rapidly, with new frameworks, techniques, and deployment patterns emerging regularly. Organizations must maintain flexibility in their architectural approaches to accommodate future innovations while protecting existing investments.
Edge computing capabilities enable algorithmic processing closer to information sources, reducing latency and bandwidth requirements while enabling real-time decision-making in distributed environments. This trend particularly impacts Internet of Things applications and mobile computing scenarios.
Automated algorithmic engineering tools increasingly handle routine aspects of model development, enabling organizations to focus on higher-value activities such as feature engineering and business problem formulation. These tools democratize access to computational learning capabilities while maintaining professional quality standards.
The integration of computational learning capabilities into modern analytical architectures represents a significant opportunity for organizations to enhance their decision-making capabilities and operational efficiency. Success requires careful attention to architectural patterns, performance optimization, security considerations, and ongoing maintenance strategies.
By following established best practices and maintaining flexibility for future innovations, organizations can build robust computational learning systems that deliver sustained business value while adapting to evolving technological landscapes.