A real-time backup network was to be set up between Berlin and Frankfurt for the West Berlin branches of a major German bank. Due to the mass of data/files and the network speeds at the time, this was difficult or even impossible. This gave rise to the idea of compressing this data before transmission. The methods known at the time were impractical in terms of the resources required (CPU time, RAM, etc.) and their consequences (paging, elapsed time, interference with other processes, etc.).

For the West Berlin branches of a German major bank, a real-time backup network between Berlin and Frankfurt was to be set up. Due to the volume of data/files and the network speeds at the time, this was difficult to impossible. This led to the idea of compressing the data before transmission. The methods known at the time were not feasible in terms of resource requirements (CPU time, RAM, etc.) and their consequences (paging, elapsed time, impact on other processes, etc.).

Tests showed that typical data from payment transactions at that time could be compressed by up to 95%, in industry parts lists even by up to 98.5%. In the case of measurement data with a wide spread of (binary) measurement values (possibly even floating point values plus mantissa), over 65% was still achieved, e.g. in the weather service. The method can be implemented extremely efficiently as a software solution for data of this type, especially for the mainframe version coded in assembler.

Tests showed that typical data in payment transactions at the time could be compressed by up to 95%, and in the industry, part lists even by up to 98.5%. For measurement data with a high variation in (binary) measurement values (possibly even floating-point values plus mantissa), over 65% compression was still achieved, e.g., in weather services. This method is highly efficient as a software solution for such data types, particularly for the mainframe version coded in Assembler.

The bank had more than 40 data centers with different platforms and database systems. Compression software was essential for the nightly synchronization of changes in these databases. This order still exists today. The range of applications within the Group has grown and continues to grow.

The bank had more than 40 data centers with different platforms and database systems. A compression software was essential for the nightly synchronization of changes in these databases. This contract remains in effect to this day. The scope of applications within the group grew and continues to grow steadily.

From 1986 to the present day, FLAM has been implemented for 60 compatible platforms. This has made it possible to declare FLAM the standard in various data exchange procedures, e.g. in clearing procedures. A large number of conversion problems had to be solved in order to ensure the compatibility of characters, fields, formats etc. as well as a transmission-compatible format of the FLAMFILE for the respective FT product/procedure. In payment transactions, this also included a format that remains printable despite compression and can be converted as a text file for file transfer (EBCDIC <=> ASCII).

In addition to the very many platforms required for the standard in payment transactions, switching systems (switches) in global mobile networks were added for the backup (local) and transmission of measurement data to central IT systems for the international clearing of measurement and billing data (billing records) from network operators and providers as well as their platforms for further processing, making FLAM the standard in the GSM network as well. FLAM quickly became widespread in the banking industry, not only among institutions, their associations and publishers, but also among service providers associated with the banking, stock exchange and credit sectors. The clearing of card operators and the handling of cards for orders and authentication were added to this.

FLAM users no longer need to know which computer system the “reader” has or will have in order to process a FLAMFILE created with FLAM. This means that FLAMFILEs can also be decompressed that were created on a system that no longer exists. When decompressing, FLAM uses the character encoding and data formatting that are standard for the respective system. It is also possible to control a converted output according to your own specifications. This also applies to archived original files from the “past of the IT world”, which are merely converted with FLAM.

From 1986 to today, FLAM has been implemented to be compatible with 60 platforms. This made it possible to establish FLAM as a standard in various data exchange processes, e.g., in clearing processes. A multitude of conversion issues had to be resolved to ensure the compatibility of characters, fields, formats, etc., as well as a transmission-ready format for FLAMFILE tailored to the respective FT product/process. In payment transactions, this also included a format that remains printable and convertible into a text file for file transfer despite compression (EBCDIC <=> ASCII).

In addition to the numerous platforms required for the payment transaction standard, there were also mediation systems (switches) in global mobile networks for securing (locally) and transmitting measurement data to central IT systems for the international clearing of measurement and billing data (billing records) of network operators and providers, as well as their platforms for further processing. This made FLAM a standard in GSM networks as well. FLAM quickly spread in the financial sector, not only among institutions, their associations, and publishers but also among service providers related to banking, stock exchange, and credit industries. This included the clearing of card operators and the handling of cards for orders and authentications.

FLAM users no longer need to know which computer system the "reader" has or will have to process a FLAMFILE created with FLAM. Thus, FLAMFILEs created on systems that no longer exist can still be decompressed. FLAM aligns its decompression with the character encoding and data formatting standards of the respective system. It is also possible to control a converted output according to custom specifications. This also applies to archived original files from the "past of the IT world" that are simply converted using FLAM.

To simplify the integration of FLAM into applications and make it completely transparent, a subsystem for accessing the DMS (Data Management System) of the MVS has been developed so that FLAM can be integrated like an I/O driver between the application and MVS (additional product). The parameter “SUBSYS=FLAM” in the DD statement is sufficient. Integration can take place in applications with sequential as well as index-sequential processing.

A major customer immediately used this for the daily processing of mass data - in the form of hundreds of VSAM files. In the process, some fundamental VSAM problems were solved en passant in favor of this access method. The effects in terms of saving CPU time, disk space and latency in a complex process were and are sensational. This is solely due to the fact that compression takes place in self-sufficient units (matrices), whereby even variable-length records are permitted. FLAM works with its own, highly optimized VSAM key management, which also allows repetitions in the application's original key.

To simplify and make the integration of FLAM into applications completely transparent, a subsystem was developed for accessing the DMS (Data Management System) of MVS. This allows FLAM to be integrated as an I/O driver between the application and MVS (add-on product). The parameter "SUBSYS=FLAM" in the DD statement is sufficient. The integration can be used for both sequential and indexed sequential processing in applications.

A major customer immediately used this for the daily processing of mass data – in the form of hundreds of VSAM files. In the process, several fundamental VSAM issues were solved in favor of this access method. The effects regarding savings in CPU time, disk space, and latency in a complex process were and still are sensational. This is solely due to the compression in self-contained units (matrices), even allowing records with variable lengths. FLAM works with its own, highly optimized VSAM key management, which also allows repetitions in the original key of the application.

A special application for encrypted data exchange required a solution in which the file to be transmitted, of any size, would not require the entire transmission to be repeated in the event of interruptions due to faults, as a fully integrable process including compression and encryption. By forming self-sufficient units, this was no problem with FLAM. You can fall back on such a self-sufficient unit that has already been compressed and encrypted and simply repeat the entire process.

A specific application for encrypted data exchange required a solution where the transmission of a large file, interrupted by disruptions, would not require repeating the entire transfer. This had to be implemented as a fully integrated process, including compression and encryption. By creating self-contained units, this was no problem for FLAM. One can rely on such a self-contained unit that has already been compressed and encrypted and only repeat the interrupted process completely.

In order to limit the amount of data to be transferred at once for large files and to make better use of parallel transmission paths, parallel and serial splitting of FLAMFILES was introduced at the customer's request when they were created.

To limit the amount of data to be transferred at once for large files or to better utilize parallel transmission paths, parallel and serial splitting of FLAMFILES was introduced at the request of customers during their creation.

It is possible to easily adapt any encryption method and key management system. The FKME (FLAM Key Management Extension) is used for this purpose. This also makes it possible to link FLAM with an existing HSM (Hardware Security Module) in the respective system. If the user needs to change the key and/or the encryption method, it is sufficient to restrict the change to the header of the FLAMFILE. The encrypted segments/members including MACs only need to be copied. A PCIDSS-compliant solution for ordering debit and credit cards, for example, was created via this service provider interface and is now in use worldwide.

It is possible to adapt any encryption methods and key management systems easily. The FKME (FLAM Key Management Extension) is used for this purpose. This also allows FLAM to be linked with an HSM (Hardware Security Module) present in the respective system. If the user needs to change the key and/or encryption method, it is sufficient to limit the changes to the header of the FLAMFILE. The encrypted segments/members, including MACs, only need to be copied. Through this service provider interface, a PCI DSS-compliant solution for ordering debit and credit cards was created, which is now used worldwide.

This procedure can be integrated into an application via the FLAM subsystem on z/OS, for example; FLAM offers the corresponding interfaces on other platforms. This provides a perfect solution that ensures that plain text data does not leave the respective application when it is created and is also not decrypted outside an application. It is therefore only ever necessary to decrypt and decompress when the plaintext data is actually needed within the respective application, without the application being aware of this. This is the most secure solution for end-to-end security. The user thus (over)fulfills the requirements of the PCIDSS (Payment Card Industry Data Security Standard).

This method can, for example, be integrated into an application on z/OS via the FLAM subsystem; on other platforms, FLAM offers the appropriate interfaces. This provides a perfect solution to ensure that plaintext data does not leave the application at the time of its creation, nor is it decrypted outside the application. Decryption and decompression only occur when the plaintext data is actually needed within the respective application, without the application being aware of it. This is the most secure solution for end-to-end security. The user meets (or even exceeds) the requirements of the PCI DSS (Payment Card Industry Data Security Standard).

With our new FLAM5 archives, we have integrated the previous full-access solution with FLAM into our archive format. The new format now represents its own incremental file system (each change leads to a new version) that works with three different keys. This allows for the separation of access to the table of contents, the index of individual members (search), and the actual data. An additional parameter string can be used to restrict access to members, formats, and columns, allowing for different views of the data. This feature facilitates secure third-party use of the data in the cloud by anonymizing certain columns with sensitive data (credit card number, personal data)..

Die neuen Archive erlauben den vollen Zugriff mit Insert, Update, Delete und Select auf einzelne Datensätze bzw. deren Spalten. Die gesamte Lösung wurde für den Einsatz in und mit der Cloud entwickelt und sichert damit den zukünftigen Nutzen von FLAM für unsere Kunden, indem wir wie bisher die Datenmengen und Datenzugriffe reduzieren, die Sicherheit erhöhen, uns nahtlos integrieren und damit die Kosten massiv senken.

With our new FLAM5 archives, we integrate the previous full-access solution with FLAM into our archive format, which now represents its own incremental file system (each change leads to a new version) that works with 3 different keys. This makes it possible to separate access to the table of contents, the index of individual members (search), and the actual data. Using an additional parameter string that can restrict access to members, formats, and columns, various views of the data can be implemented. This serves, for example, to enable secure third-party use of data in the cloud by anonymizing certain columns with sensitive data (credit card numbers, personal data).

The new archives allow full access with insert, update, delete, and select operations on individual records or their columns. The entire solution was developed for use in and with the cloud and ensures the future utility of FLAM for our customers by continuing to reduce data volumes and data access, increasing security, integrating seamlessly, and thereby massively reducing costs.