FLAM® Issue Tracker

FL5 - Change Log

FL5 - 5.1.27 (Released 2022-05-16) View Issues ]


* More usage of regular expressions
** Support new command GREP in FLCL
*** Use this new command in CLIST FLGREP
** Make FMTERR usable for TVD format
** Correct match and replace statistics
** Add more context to error messages
* Use zACS in QA process of FLAMSTC (no findings, only 0C4 possible)
* Use character conversion without pre-calculation (RawRun) only if no factory to reuse the handle available
* Add NoRun switch to FLAM and ICNV commands
* Support new SAF policies LOG.INPUT and LOW.ENTROPY
** LOG.INPUT to prevent the logging of input parameter strings (could contain credentials (passwords, keys, PINs))
** LOW.ENTROPY to prevent the use (environment variable FL_ALLOW_LOW_ENTROPY) of low entropy for random values
* Add new chapter for security considerations to manuals

* Bugfixes
** Fix use of free'ed memory in table support concerning default ccs_id default delimiter if directory walk used
*** if no CCSID or delimiter for the columns/elements defined, then the determined default value was lost till the second file
*** The default value was stored in the FMTTAB handle which is closed and the pointer then points to already freed memory
*** The result are too long or wrong CCSIDs or damaged delimiters
*** The problem could be solved by defining the CCSIDs and element delimiters
** Correct compression mode in member list for FLAMFILEs on UNIX/Windows (for MODE=ADC and old FLAM encryption)


ATTENTION: For character set conversion, IBM Unicode Services are used on z/OS
by default if possible and if it improves conversion speed. In a few cases, the
behavior for incomplete or non-convertible characters will differ from the old
FLAM behavior. To prevent this change in behavior, you can set the system symbol
&FLIBMUS to OFF, but this will result in more CPU and memory utilization for simple
character conversions (see chapter 'Use of IBM Unicode Services' for more

ATTENTION: Incorrect handling of surrogates in the character conversion module was
fixed for UTF-16LE. This fix changes behavior and breaks backward compatibility.

 * Regular expressions (Perl-compatible) can now be used for:
 ** Generating output file names from input filenames (Input to Output Name Mapping)
 ** Validation or replacement of string elements
 ** Validation or replacement of string records during text formatting
 ** Validation or replacement of string records during binary formatting for records
 ** Validation or replacement for columns in table support
 * A tool to test regular expressions is available as object run.regexp() of the UTIL command
 * A new CLIST named FLGREP is available to search for a regex pattern using the directory walk on z/OS
 * About and version information was reworked
 * Rework handling of environment variables and add new built-in functions HLPENV and LSTENV
 * Complete rework of character conversion module
 ** Character conversion now differentiates between illegal characters (EILSEQ) and malformed encoding (ENOMSG)
 ** Uses the same conversion handle if user table and report file are the same
 ** Uses fast conversion algorithms in more cases
 ** Reduced memory utilization and increased overall performance
 * TR instruction is used on SystemZ for single byte to single byte character translations in character conversion module
 ** Significantly increases performance and reduces CPU utilization compared to IBM Unicode Services
 ** Substitution or identity mapping must be activated as error handling (CHRMODE=STOP will be 10x slower)
 ** A new environment variable (FL_DEFAULT_CHAR_MODE (&FLCHARM)) changes the default CHRMODE from STOP to SUBSTITUTE
 * IBM Unicode Services is used if possible to lower memory and CPU utilization
 ** If none of the special features of FLAM character conversion module are requested, Unicode Serivces with HW support are used by default
 ** A new environment variable (FL_IBM_UNICODE_SERVICE (&FLIBMUS)) controls the use of Unicode Services
 ** See new chapter 'Use of IBM Unicode Services' for more information
 * New parameter added to keep byte order marks (KEEPBOM) for faster character set conversions
 ** This brings BOM handling more in line with IBM Unicode services
 ** An invalid BOM will now cause an error with UTF-8 and UCS-1
 *** Unicode services keep an invalid BOM in the middle of the data
 *** To detect this and the wrong data behind it, the error is required
 * Re-implementation of usage logging to support eligible SRB(zIIP) and zIIP times
 * Support to dump elements in write.xml()
 * New version of LZMA SDK (21.7) including x86 optimization
 * zlib updated to version 1.2.12 (CVE-2018-25032)
 * zstd updated to version 1.5.2
 * libbzip2 updated to version 1.0.8
 * A chapter regarding performance considerations was added to manuals
 * Loading the environment occurs a little earlier than before due to character conversion optimizations
 * Protect fliconv (FLCICV) interface with feature code (FLAC_FLCICV)
 * Bugfixes
 ** Fix statistics values in FLAMFILE I/O (FIO-Time and Block-Count)
 ** Fix wrong handling of surrogates for UTF-16LE
 *** Encoding of 4 byte characters (codepoints >= 64k) was wrong
 *** Such a 4 byte encoding must be 2 times 16 bit little endian integers and not a 32 bit little endian integer
 ** Fix logging if self-signed PGP certificate are imported
 ** Correct handling of dirty delimiter (0x0D...0DC285) at text parsing
 ** Don't write log warning for attribute length mismatch more than once
 ** Correct pointer and space calculation after realloc in character conversion module (CNVCHR)
 ** Print info instead of warning on attribute length mismatch when reading in FMT.REC()
 *** Print only one log message for attribute length mismatches and not for each record
 ** Correct sample program SOFLCICV and SPFLCICV if output buffer too small
 ** Fix a missing initialization in CNVREC

- 0000783: [2.2 Subprogram FLUC (CONV)] Add match of regular expression to string converter (Falk Reichbott) - resolved.
- 0000962: [1.1 FLCL] Limit the maximal internal block size to prevent memory exhausting through expansion at decompression (Falk Reichbott) - resolved.

[2 issues]

Copyright © 2000 - 2024 MantisBT Team
Powered by Mantis Bugtracker