CLE/P - Change Log CLE/P - 1.4 (Not Yet Released) [ View Issues ] ============================== - 0001013: [1. CLE] Make SILENT and QUITE switch available by environment variables (Falk Reichbott) - resolved. - 0000910: [2. CLP] Support dynamic system symbols of zOS (Falk Reichbott) - resolved.
[2 issues]
CLE/P - 1.3 (Released 2019-12-22) [ View Issues ] ================================= - 0000839: [4. Documentation] Support a new kind of documentation generation without redundancy (Falk Reichbott) - resolved. - 0000159: [1. CLE] Make built-In functions ABOUT, VERSION and LICENSE invisible if a corresponding string is NOT available (Falk Reichbott) - resolved. - 0000945: [2. CLP] Support escape sequences for dia critical characters in EBCDIC in CLP strings (Falk Reichbott) - resolved. - 0000414: [2. CLP] Support of dynamic length strings and arrays (Falk Reichbott) - resolved. - 0000538: [2. CLP] Support for constant and variable expressions including units (KiB, MiB, TiB) (Falk Reichbott) - resolved.
[5 issues]
CLE/P - 1.2 (Released 2016-12-23) [ View Issues ] ================================= - 0000876: [2. CLP] Support CCSID for command and property strings (Falk Reichbott) - resolved. - 0000152: [2. CLP] Add new data type for options (-R, --Recursive) (Falk Reichbott) - resolved. - 0000748: [2. CLP] Support read from files for strings to protect passwords (Falk Reichbott) - resolved. - 0000683: [z. Other] Comment out properties with default values when generating property files (Falk Reichbott) - resolved. - 0000684: [2. CLP] Error messages with "longer than n" are wrong (Falk Reichbott) - resolved. - 0000614: [2. CLP] Wrong error position, if semantic error (Falk Reichbott) - resolved. - 0000564: [1. CLE] DD names don't work for output files (Falk Reichbott) - resolved. - 0000562: [2. CLP] Error messages for property parsing are not correct (Falk Reichbott) - resolved. - 0000561: [2. CLP] Definition of properties over environment variables results in strange error messages. (Falk Reichbott) - resolved. - 0000560: [2. CLP] Properties don't work, if string and keyword possible (Falk Reichbott) - resolved. - 0000536: [1. CLE] Cannot run multiple flcl processes in parallel if property files of different owners are used (Falk Reichbott) - resolved. - 0000554: [1. CLE] If selection flag is set, print not the right type, because secelction, allowes only selection values(strings) (Falk Reichbott) - resolved. - 0000548: [3. Interface] Remove global and static variables (Falk Reichbott) - resolved. - 0000546: [3. Interface] CLP and CLE can only be linked static, DLL link required (Falk Reichbott) - resolved. - 0000534: [1. CLE] Help function don't show the file properties (Falk Reichbott) - resolved. - 0000533: [2. CLP] Properties can only be shown for overlays and object but not for single parameters (Falk Reichbott) - resolved. - 0000532: [2. CLP] add parameter for disabling aliase in help-parameter list (Falk Reichbott) - resolved. - 0000530: [1. CLE] List only defined properties (Falk Reichbott) - resolved. - 0000531: [1. CLE] Strore config and property files as default in the current home directory (Falk Reichbott) - resolved. - 0000528: [1. CLE] A built-in function to change property values would be fine (Falk Reichbott) - resolved. - 0000523: [1. CLE] The license text provided to CleExecute are not given to the run function like the about or version string (Falk Reichbott) - resolved. - 0000522: [1. CLE] OID for command overlays not known at mapping (Falk Reichbott) - resolved. - 0000521: [2. CLP] Support object without parenthesis and overlays without dot (Falk Reichbott) - resolved. - 0000520: [1. CLE] Support of default command or built-in function (Falk Reichbott) - resolved. - 0000346: [2. CLP] Selections for aliases dose not work (Falk Reichbott) - resolved. - 0000498: [2. CLP] Allow keywords preceded by '-' or '--' (Falk Reichbott) - resolved. - 0000432: [5. Deployment] Replace sub revision number bei build number (Falk Reichbott) - resolved. - 0000401: [2. CLP] If path empty ("") at help and syntax functions an error occured (Falk Reichbott) - resolved. - 0000331: [1. CLE] Add STDENV DD statement support to CLE batch utility to simply define enviroment variables. (Falk Reichbott) - resolved. - 0000319: [2. CLP] Define properties/defaults over enviroment variables (Falk Reichbott) - resolved. - 0000261: [2. CLP] Support of property or command line only parameter (Falk Reichbott) - resolved.
- 0000547: [1. CLE] Parameter files don't work anymore (Falk Reichbott) - resolved. - 0000544: [2. CLP] Segmentation fault if error in parameter file (Falk Reichbott) - resolved. - 0000541: [1. CLE] Enviroment variable to define the config file don't work properly (Falk Reichbott) - resolved. - 0000505: [4. Documentation] Do not print manpage twice (Falk Reichbott) - resolved. - 0000494: [1. CLE] Argument list missing if help used with man keyword (Falk Reichbott) - resolved. - 0000488: [2. CLP] Prevent logging and tracing of passwords (Falk Reichbott) - resolved. - 0000476: [2. CLP] Improve string handling for different shells and platforms (Falk Reichbott) - resolved. - 0000239: [1. CLE] If the RUN function of a command failed then the FIN function must be done (Falk Reichbott) - resolved. - 0000232: [2. CLP] Use of ' in strings or " in suplements are not possible (Falk Reichbott) - resolved. - 0000231: [1. CLE] Function to clear configuration data required to ensure a correct setup (Falk Reichbott) - resolved. - 0000158: [1. CLE] Add new built-in function to display the current license (Falk Reichbott) - resolved. - 0000153: [3. Interface] Add flag to disable the numbering for generated documentation (Falk Reichbott) - resolved. - 0000182: [1. CLE] Set of enviroment variables over the config file sometimes does not work (Falk Reichbott) - resolved. - 0000150: [4. Documentation] Separate doxygen interface specification of CLE/P for publication (Ulrich Schwab) - resolved. - 0000133: [3. Interface] Add flag to CLE command table to mark commands as hidden (Falk Reichbott) - resolved.
- 0000665: [4. Element, Byte, Record or other API] Add function to map condition code in a message to FLUCUP (Falk Reichbott) - resolved. - 0000295: [3. Subsystem, Driver, Plugin, SPI] Element interface for original data (Falk Reichbott) - resolved. - 0000592: [2.2 Subprogram FLUC (CONV)] Support of round trip conversion and identity conversion as error handling for invalid characters (Falk Reichbott) - resolved. - 0000824: [2.2 Subprogram FLUC (CONV)] Extend table support for IMS/DB2 unload formats (more than one table format per file) (Falk Reichbott) - resolved.
* Support index definitions for arrays in XML of table support at read
** With ‘[n]’ or ‘(n)’ at the end of a tag
** You can assign an index starting with 1 to it in order to access a specific repetition in arrays
** The indexing at write ensure grouping (prefixing with '!' and '|' is then not required anymore but still supported)
5.1.28-33163
* Add new parameter RENAME for local or remote rename of files after successful processing
5.1.28-33154
* Fix bug writing corrupted FLAMFILE when an empty file is encrypted
* Improve statistics for generation of signature file
* Support oldest signature procedure as default in FLM4SEPA
** Adjust docu (don't migrate after update or delete)
5.1.28-33135
* Add new parameter (EMPTY/EMPTYFILE) for empty file handling at write
** INFO - to write an informational to the log
** WARNING - to write a warning to the log (completion code 1)
** ERROR - Stop processing with an error (completion code 8)
** DELETE - Delete the empty file after processing
** This is implemented for flat files (BLK, REC, TXT) but not for archives (ZIP, FLAM, ...)
* Fix few issues concerning the CPACF deflate instruction (ZEDC)
** Don't compare local general purpose flag word in ZIP archives with central one
** Correct Handling of GZIP header with ZEDC usage and improve log messages
* FLCC fixes
** fix editing of overlay commands
** override keyboard search of CLP tree view avoid unexpected input focus change
5.1.28-33104
* Fix endless loop in record interface if block orientation used (size(0)*count(reclen)=0)
* Support error trace for errors in stub routines of record interface
* Adjust internal state if SKIPEQUAL done in character conversion set the CCSID for the data
** Prevent wrong data handling if character conversion was skipped
* Use gmtime_r instead of localtime_r for dostime conversion in ZIP archives
** prevents different modification times in file header
* Use "wb" instead of "wbs+" for streamed ZIP formats in YIOOPN
** "wb+" is not supported by some SSH/SFTP server and not required for streamed ZIP files
** For such server the streamed format will now work
* Fix charcter conversion expansion handling in final run
** Prevent remaining rest error at expansion of conversion table from 64kib to 2MiB code points at UTF conversion
* fix incorrect processing of SSH stat() data that prevented directory deletes via SSH
* Make primary and secondary space for signature file definable over environment variables at FL4SEPA
** The migrate function deletes the signature file and allocate it with the used space but this can result in SB37 at a large append
** To allocate more the environment variables FL4SEPA_SIGFILE_PRIMARY_SPACE_TRK and FL4SEPA_SIGFILE_SECONDARY_SPACE_TRK can be used
* Correct condition to prevent migration of signature file if not required (same version) at FL4SEPA
5.1.28-33083
* Fix 0C4 at directory walk with SSH
** The handle was invalid after sftp_closedir() even on error
5.1.28-33071
* Support grouping at XML write in table support
** Currently, the exclamation mark and the pipe symbol are equally
supported to define the target level in the path specifications
(see Build 5.1.28-32984).
** With this build, the target level is only reverted to if there has
been a group change, i.e. the elements do not belong to the same
group.
** Multiple use of the exclamation mark or the PIPE symbol can be used
to define groups in arrays.
** This build thus makes the behavioral change of build 5.1.28-32984
if you only use the exclamation mark or the pipe symbol.
** If you do this alternately, you will have reproduced the behavior of
build 51.28-32984.
** You can find more information on this in the documentation.
5.1.28-33062
* Fix damaged ZIP files if ZEDC produce blocks smaller the 8
** Normal zlib don't result in this error
** only ZEDC with a small last block of compressed data
** Add several plausichecks and improve logging an error handling
* Fix several memory overlapping issues with memcpy
** ICV module if __tr instructions used on z/OS
** For length field handling in table support
** Masking processor in table support
** In CMPMOD module of FLAM4 record interface
** In strlcpy of CLEPUTL module of FLAMCELP
** Return data incomplete error if end of stream not achieved at final run at decompression
* Several small documentation corrections and improvements
** Standard ways to define environment variables on the different systems
** Sample to use FLUCSUB on z/OS with IEBGENER
5.1.28-33038
* Fix remaining zACS issues
* Don't check license of FMTTAB if internally used
** Allow use of GREP command also for customer don't have the table support available
* Ensure set of PWD/KEY in directory walk and produce only a warning if a key is provided but the FLAMFILE is not encrypted in FIO.FL4()
** The old FLAM for Windows produce encrypted FLAMFILEs with wrong encryption mode (if HEADER=YES and CRYPTOMODE=FLAM)
** If directory walk used for FLAMFILE members the FLAMFILE is open by directory walk but no PWD/KEY was set
** The open handle are then used to read the FLAMFILE member but the key was missing
5.1.28-33028
* Add new parameter HEADLN at read to table support
** Can be activated to read over an head line in CSV files
** Prevent the requirement to compare the column names to detectd a head line
* Add new parameter FLATTX at write of XML in table support
** Prevent fallback to root level for each transaction
** Can be used to produce flat XML lists
* Support GMOFFABS and LCOFFABS as key words in CLEP strings
** will be strings containing the difference to GMT in the form 'HH'
* Support GMOFFSET and LCOFFSET as key words in CLEP strings
** will be strings containing the difference to GMT in the form '+HH' or '-HH'
* Add plausibility check in FMT.TXT for CCSID used by regular expression
** The character set for the data must match the character set derived from the REGEXP-CCSID
* Enable format string of newMember() in Java byte interface to be null
** Increase significant performance if the same format used for each member
5.1.28-33014
* Rework FLM4SEPA to prevent mismatch errors
** Add plausibility check if append used for archives
** Use correct identifier for member names at signature file generation
*** The first identifier was also used for the next members
*** Now the identifier of the current member name of the transaction file
is used for the currents member header of the signature file
** The archive clusters are only open if records loaded
*** If no records provided to the append operation, then the FLAMFILEs are not touched
*** On initial load this can be enforced and will done for append by default
*** You can also enforce the open for append using the return code values as input
** Do implicit migrate of signature files to KSDS for update and delete
*** If file organization or record format mismatch
*** If version 23 used but still a PS allocated as signature file
then the migration to a KSDS will now also be done
*** In previous version the an error occurred if file organization or
record format does not match with the version
* Fix zACS findings for FLAMSTC
** Add eye catcher to XMR routines to prevent 0C4 for random input data
* Fix use after free if PC call failed in LE-less interface on z/OS (FLUCSUB)
5.1.28-33004
* Add new appendix for FLKOMP and FLDECO utilities (version 1 of FLAM) to FLAM4 manual
* Add support for different kind of namespace handling at read of XML
* Add use of literal cache for XML tags to FMT.XML of XCNV at read
* Bugfixes
** Fix array and child handling issues in FLCC
*** use correct index to insert new element
*** avoid removing last remaining element
*** remove redundant checks
*** do not load manpage with click on add or remove column
*** never show childs of overlay when no child is selected
** Add again a missing free's for memory in remote (SSH based) directory walk
** Improve error handling in remote directory walk if stat() failed
** Fix missing member header after update in place in the first matrix
5.1.28-32984
* Add support for exclamation mark in XML format of table support at write for all columns
** Enforce to write to a lower level if the path does not require this
* Bugfixes
** Add missing free for memory in remote (SSH based) directory walk
*** Could result in segmentation fault or memory allocation failed if a lot of files used
** Fix several issues in FLAM-VASAM using ADC
*** Fix 0C4 in FLMGKY for an empty FLAMFILE VSAM KSDS
**** Read over a matrix only if the matrix is not empty
*** Fix 1 byte shift after FLMGKY failed with invalid key
*** Fix ABEND4038/02 if FLMPKY/IKY used for ADC VSAM-KSDS-FLAMFILE
*** Correct statistics values in FLMCLS/FLU
** Correct error message from FKM5 in PGP component if signing key not found
** Fix wrong handling of system symbols and environment variables
*** With version 5.1.27 a central table for environment and system variables was introduced
*** The loop to run over this table was buggy at initial handling and for the system information
*** The default behavior for ENVID (key labels) was changed and wrong
5.1.28-32966
ATTENTION: Users of the Java Byte Interface must also upgrade their Java library
after upgrading to this or a new version of FLAM. Failing to do so will render
your Java code inoperable. This is necessary due to the significant changes
to add 31 interoperability with MVS FLAM from 64 Bit Java. You do not need to
change your existing Java code. The new Java library can be downloaded here:
https://flam.de/en/download/addons/Java/ByteInterface/
* Update of XML parser libexpat to 2.4.9 (fixes CVE-2022-40674)
* Add flat edit mode to Frankenstein Limes Control Center (FLCC)
** Simplifies handling of complex objects
** May be helpful for people with disabilities
* The 31 bit MVS version of FLAM can now be used from a 64 Bit Java VM on z/OS
** A new FLCBYTJ3 JNI DLL was added to support this which is automatically loaded
by the NativeLibraryLoader class if the JVM was configured correctly.
** See JZOSFCBI.jcl in JOBLIB how to use it
** A mandatory update for the Java Byte Interface JAR file is available as separate download
* Add new SAF policies to control authentication mechanism for SSH connections and use of custom config file
** SSH.AUTH.PUBLICKEY.ALLOWED - SSH authentication with public keys (READ)
** SSH.AUTH.INTERACTIVE.ALLOWED - SSH authentication with GSS API (UPDATE)
** SSH.AUTH.GSSAPI.ALLOWED - interactive SSH authentication (UPDATE)
** SSH.AUTH.PASSWORD.ALLOWED - SSH authentication with password (UPDATE)
** SSH.AUTH.HOSTBASED.ALLOWED - only host based SSH authentication (CONTRL)
** SSH.AUTH.NONE.ALLOWED - none SSH authentication (ALTER)
** SSH.AUTH.UNDEFINED.ALLOWED - undefined SSH authentication (ALTER)
** SSH.CUSTOM.CONFIG.FILE.ALLOWED - A custom configuration file is not allowed (CONTRL)
* Make reuse of a SSH connection dependent of the PCAP file name
** Use different SSH connections if PCAP enabled or not
* FLGREP has been corrected so that it now displays the search results
even if there were errors in one or more files.
* Adjust documentation of supported formats and algorithms
* Bugfixes
** Add missing free for memory if hash check file used at write
- 0001028: [4. Element, Byte, Record or other API] Support 64->31 bit interoperability for 64 bit java application using 31 bit FLAM MVS byte interface (Falk Reichbott) - resolved.
* More usage of regular expressions
** Support new command GREP in FLCL
*** Use this new command in CLIST FLGREP
** Make FMTERR usable for TVD format
** Correct match and replace statistics
** Add more context to error messages
* Use zACS in QA process of FLAMSTC (no findings, only 0C4 possible)
* Use character conversion without pre-calculation (RawRun) only if no factory to reuse the handle available
* Add NoRun switch to FLAM and ICNV commands
* Support new SAF policies LOG.INPUT and LOW.ENTROPY
** LOG.INPUT to prevent the logging of input parameter strings (could contain credentials (passwords, keys, PINs))
** LOW.ENTROPY to prevent the use (environment variable FL_ALLOW_LOW_ENTROPY) of low entropy for random values
* Add new chapter for security considerations to manuals
* Bugfixes
** Fix use of free'ed memory in table support concerning default ccs_id default delimiter if directory walk used
*** if no CCSID or delimiter for the columns/elements defined, then the determined default value was lost till the second file
*** The default value was stored in the FMTTAB handle which is closed and the pointer then points to already freed memory
*** The result are too long or wrong CCSIDs or damaged delimiters
*** The problem could be solved by defining the CCSIDs and element delimiters
** Correct compression mode in member list for FLAMFILEs on UNIX/Windows (for MODE=ADC and old FLAM encryption)
5.1.27-32770
ATTENTION: For character set conversion, IBM Unicode Services are used on z/OS
by default if possible and if it improves conversion speed. In a few cases, the
behavior for incomplete or non-convertible characters will differ from the old
FLAM behavior. To prevent this change in behavior, you can set the system symbol
&FLIBMUS to OFF, but this will result in more CPU and memory utilization for simple
character conversions (see chapter 'Use of IBM Unicode Services' for more
information).
ATTENTION: Incorrect handling of surrogates in the character conversion module was
fixed for UTF-16LE. This fix changes behavior and breaks backward compatibility.
* Regular expressions (Perl-compatible) can now be used for:
** Generating output file names from input filenames (Input to Output Name Mapping)
** Validation or replacement of string elements
** Validation or replacement of string records during text formatting
** Validation or replacement of string records during binary formatting for records
** Validation or replacement for columns in table support
* A tool to test regular expressions is available as object run.regexp() of the UTIL command
* A new CLIST named FLGREP is available to search for a regex pattern using the directory walk on z/OS
* About and version information was reworked
* Rework handling of environment variables and add new built-in functions HLPENV and LSTENV
* Complete rework of character conversion module
** Character conversion now differentiates between illegal characters (EILSEQ) and malformed encoding (ENOMSG)
** Uses the same conversion handle if user table and report file are the same
** Uses fast conversion algorithms in more cases
** Reduced memory utilization and increased overall performance
* TR instruction is used on SystemZ for single byte to single byte character translations in character conversion module
** Significantly increases performance and reduces CPU utilization compared to IBM Unicode Services
** Substitution or identity mapping must be activated as error handling (CHRMODE=STOP will be 10x slower)
** A new environment variable (FL_DEFAULT_CHAR_MODE (&FLCHARM)) changes the default CHRMODE from STOP to SUBSTITUTE
* IBM Unicode Services is used if possible to lower memory and CPU utilization
** If none of the special features of FLAM character conversion module are requested, Unicode Serivces with HW support are used by default
** A new environment variable (FL_IBM_UNICODE_SERVICE (&FLIBMUS)) controls the use of Unicode Services
** See new chapter 'Use of IBM Unicode Services' for more information
* New parameter added to keep byte order marks (KEEPBOM) for faster character set conversions
** This brings BOM handling more in line with IBM Unicode services
** An invalid BOM will now cause an error with UTF-8 and UCS-1
*** Unicode services keep an invalid BOM in the middle of the data
*** To detect this and the wrong data behind it, the error is required
* Re-implementation of usage logging to support eligible SRB(zIIP) and zIIP times
* Support to dump elements in write.xml()
* New version of LZMA SDK (21.7) including x86 optimization
* zlib updated to version 1.2.12 (CVE-2018-25032)
* zstd updated to version 1.5.2
* libbzip2 updated to version 1.0.8
* A chapter regarding performance considerations was added to manuals
* Loading the environment occurs a little earlier than before due to character conversion optimizations
* Protect fliconv (FLCICV) interface with feature code (FLAC_FLCICV)
* Bugfixes
** Fix statistics values in FLAMFILE I/O (FIO-Time and Block-Count)
** Fix wrong handling of surrogates for UTF-16LE
*** Encoding of 4 byte characters (codepoints >= 64k) was wrong
*** Such a 4 byte encoding must be 2 times 16 bit little endian integers and not a 32 bit little endian integer
** Fix logging if self-signed PGP certificate are imported
** Correct handling of dirty delimiter (0x0D...0DC285) at text parsing
** Don't write log warning for attribute length mismatch more than once
** Correct pointer and space calculation after realloc in character conversion module (CNVCHR)
** Print info instead of warning on attribute length mismatch when reading in FMT.REC()
*** Print only one log message for attribute length mismatches and not for each record
** Correct sample program SOFLCICV and SPFLCICV if output buffer too small
** Fix a missing initialization in CNVREC
- 0000783: [2.2 Subprogram FLUC (CONV)] Add match of regular expression to string converter (Falk Reichbott) - resolved. - 0000962: [1.1 FLCL] Limit the maximal internal block size to prevent memory exhausting through expansion at decompression (Falk Reichbott) - resolved.
* Support OpenSSL-compatible encryption and decryption of data streams (openssl enc)
* Support Zstandard (zstd) for stream compression and as compression format in ZIP archives
* Support xz compression format in ZIP archives
* SHA-3 hardware acceleration is now also supported on z/Linux (if available)
* SSH connections on z/OS, USS and z/Linux use HW-acceleration by default (if available) for:
** Data encryption: AES (modes: CBC, CTR, GCM)
** HMAC: MD5, SHA-1, SHA-2
** On other platforms, HW-acceleration is implemented by OpenSSL (if available)
* Add count and mask to log output for literal cache statistics
* Add missing environment variables to system info
* Correct help message for RECDELIM parameter of FLAM command
* Improve MP4 detection and simplify and unify magic byte handling
* Improve record handling concerning suppression of padding bytes
** Don't suppress padding in control characters if PRNCONTROL=RETAIN when reading
** Suppress padding before adding control character or key values for writing of records
** Don't suppress trailing whitespaces for text in record I/O (only suppress padding)
* Write text records independent of the local character set (delimiter and keys)
** Support insert of key values (KEYDISP=NEW) in target character set
* Don't use x'0D25' for EBCDIC on Windows as default delimiter anymore
** x'15' is now used to build the default text delimiter for writing of text records
* Support suppression of control character in FIO.TXT()
* Support replacement of backspace characters in FMT.TXT()
* If RPLHTB/RPLTAB is defined without a value 8 is used by default
* If RPLVTB is defined without a value 1 is used by default
* If parameter RPLFFD (replace form feed) is defined without a value, the default is set to 60 lines
* New version (250) of FL4SEPA which uses new bloom filter of FL5 project
* Increase the default record count to ensure a minimal block size of 64KiB in FIO.REC/FL4()
* FLCC:
** Improve parameter file handling:
*** Use name of parameter file in open of object/overlay if it is set
*** Show changes as such when object/overlay is opened
*** Show error when specified parameter file cannot be used and provide file dialog to select an alternative
If one is selected by the user, change the name of the parameter file for the object/overlay
** Simplify config management:
*** all configurations are now accessible in one option dialog
*** Add new option to use relative paths in command line
*** when adding a variable to the flcl config file
environment variables used in FLCL are presented as completion while typing the name.
** Extend help menu
*** add 'use cases' dialog to select and load examples from FLCL manual
*** add entry to get system information and allow copy to clipboard
*** add entry to open flam issue page in web browser
** fixes:
*** disable array element add function for inactive objects/overlays
*** avoid crash on open of context menu when no current line is set
*** avoid text cropping in tree view
** general:
*** save 'expand all' option on exit and use it in parameter file windows as well
*** overlays will now re-use already entered values where possible
* Set default for heap segment management on z/OS to FREE
** Improve memory efficiency by releasing allocated storage segments where possible
* Improve defaults if data format and data type is not provided in state string
* Improve defaults for block size and record count in byte and record interface
* Improve defaults for writing of FLAMFILEs if split of blocks in records is required
* Add new switch PERFILE to FALLLOC object when writing to enforce close and open for each file in the directory walk
* Improve description of writing several files in directory walk into a generation data group on z/OS
** infile='/path/to/*.txt' pattern='<SYSUID>.GDG(+[ind0])' outfile='USER.GDG(+n)'
whereby n starts with 1 and is incremented for each file
* Use FIX/MAXLEN as default for VAR format (to simplify conversion from FIX to VAR format)
* Implement a cache for headers in table support to significantly improve performance on table changes
* Add new SAF policies to control minimum length of PGP passphrases (PGP.PWD.LEN.ENC/DEC)
* Check the first 2 bytes (PK) of a ZIP archive to show better error message if no directory trailer is found
"no ZIP file or truncated ZIP file"
* Improve message if member of archive was used but no archive was detected and the member access failed during decompression
* Verify at formatting that no member access is pending when reading (indicates that an archive access was not successful)
* Support error messages from PKCS11 devices and improve error handling of all crypto components
* Increase the default maximal record count from 1024 to 16384 to ensure significant block size even for very small records
to prevent empty blocks after decompression and wrong detection caused by missing data
* Ensure secure erase of key values and passwords if not longer required in memory
* Bugfixes
** Provide BZIP2 and XZ member lists in info command
** If the first file list did not have a match, remaining file specifications were ignored and a 'not found' error occurred
now the error occurs only when no file specification has a match
** Properly close PKCS#11 implementations to prevent sporadic error 401 (CKR_CRYPTOKI_ALREADY_INITIALIZED)
** Define SAF error level as default to 8 to prevent errors of RACF definitions incomplete
- 0001024: [1.1 FLCL] If the first file list does not have an match the remaining file specifications are not used and a error (not found) occure (Falk Reichbott) - resolved. - 0000774: [1.2 FLCC] Support of use cases as simplification of FLCC commander (Ulrich Schwab) - closed.
* Fix block change issue in name space support in TABXML
** In certain cases the URI was lost after a block border
** Then the match of XML tags failed and an unexpected format error occurs
* Prevent memory gap for XML element list if namespace support used
* Improve error messages in table support (more data at error position)
* Print statistics about literal cache usage to log (to adjust size (see below) for table support)
* Add new environment variable (FL_LITERAL_CACHE_SIZE) to define size of hash table for literal cache
** In table support using XML we recommend to increase the size from default 12 to 16
** e.g. FL_LITERAL_CACHE_SIZE=16
* Correct QUIET and SILENT for built-in functions of FLCL
* Add anti virus scanning check (EICAR) to system information (flcl info get.system)
5.1.25-30818
* Add namespace handling for XML <-> table transformations
** defaults(...xmlns(prefix='ns0' uri='http://www.w3.org/1999/xhtml')...)
** Propagate and reset header values over row borders (this was a general bug)
** Correct reset logic for optional values inside of a transaction
*** If different row definitions used, which starts with the same path, the header values must set and reset for each of the tables
* Include level 0 for flat XML files when initializing a table
** Supports more than one XML format in one file
** Improve documentation for XML <-> tables conversions
* Add new CLIST 'FLSYS' to easily determine system information for the use of FLAM on ISPF
* Fixed hardware crypto support (AES) on zLinux
* Fixed BUS_ERROR on SPARC when converting from single byte character sets to UTF-16/32
* When writing via fio.blk/rec/txt(), the modification time can be copied from the original file by setting modtime=ORIGINAL
* SHA-3 hash calculation now uses CPACF instructions on z/OS / USS if available
* Use new libxz and liblzma
5.1.25-30684
* New internal crypto/hash library with reduced overhead and more flexibility
** SHA-3 support added (via OpenSSL, hardware support is work-in-progress)
** Optimized CRC-32 and CRC-32C software implementations, faster by factor 5 or more (depending on CPU)
** CRC-32C makes use of SSE4.2 hardware acceleration where available (x86 CPUs)
*** Faster by up to factor 30 (>10GiB/s) compared to old software implementation
** Fixed CRC-64-JONES implementation (little endian platforms produced incorrect results)
** Fixed CRC-40-GSM (produced incorrect results)
** Log now shows whether hardware, OpenSSL or custom software implementation of cipher or hash algorithm is used
** Random number generator (RNG) can fail if the operating system offers no source of entropy
*** Some PGP operations may fail as a result if no entropy is available
*** Environment variable FL_ALLOW_LOW_ENTROPY can be set to allow a low entropy source which leads to weak encryption
* OpenSSL updated to 1.1.1j
** OpenSSL is no longer linked statically, resulting in a significant reduction of the binary size
* libexpat updated to 2.4.1
** Adds protection against billion laughs attacks (CVE-2013-0340/CWE-776) and other fixes
* LZMA implementation updated to version 19.00 from the LZMA SDK
* Improved tracing capabilities of FLAMSTC, FLAM subtask and FLUC subsystem
* Use application-specific runtime option defaults on z/OS to prevent stack overflows and heap storage problems
** See hlq.FLAM.SRCLIB(CEEUOPTx) for the used options, DD:CEEOPTS can still be used for redefinition
* Improved handling of SSH public key authentication with custom key files
** In previous versions, both a public and private key had to be specified for public key authentication with a custom key
** Now only the private key is mandatory
** The public key is computed from the private key if only a private key is specified via net.ssh() and the file <privatekeypath>.pub does not exist
** Security implication: The private key must be decrypted even if the server rejects the public key
** Documentation for net.ssh() regarding public key authentication was updated
* Significant performance increase of character conversion when converting from single byte character set or from UTF-8
** If MODE=SUB/IDT/XLT used and all possible 256 single byte always converted to one, two or four bytes
** and if the report file not used, then special mapping functions are used to reduce CPU by factor 4
** To ignore non convertible character or to stop at a wrong character (default behavior) or to report manipulations a special handling per byte is still done
** but in all other cases if the output length constant the special faster functions to write always 1, 2 or 4 bytes are used
** The change includes also support for UCS1 like UTF8 but with invalid codepoints greater 65535
*** Till 5.1.25 UCS1 was implemented as UTF8 because the ICU package (used to generate our code) does the same
** To use the optimisation also for UTF16/32 the BOM character are now the limit for conversion table pre-caculation
*** For codepoints greater BOM the expansion of the mapping table to 2 mega of entries is done
*** and the ignore of BOM and the character for nothing will result in the old handling per character in this case
* Increased performance when binary formatting is used (FMT.BIN()) if no attribute handling is required
* Improved I/O performance
** Don't use minimal block size at static allocation if no block size was specified
** Reduced CPU consumption of record I/O code (including FIO.DAT())
* Improved determination of default space on z/OS if fixed record format used to allocate space for padding bytes
** And multiply the expansion if a record length used which is greater than the original record length
* Statistics output now provides minimum and maximum length of processed records in record I/O
* Extended string notation (e.g. a'xyz' for ASCII string, x'F1A5' for hex values) is now supported for any string by CLP
** The local character to ASCII/EBCDIC and hex to binary conversion can also be used for zero terminated strings
** If the string not mark as binary a binary zero inside the strings is not allowed
* Support for record formats TXT-ASA/MCC and DLM-ASA/MCC added
** Required to copy list datasets written to USS pathnames (e.g. compile and link) from USS to MVS datasets
** Few tools used on USS writes ASA/MCC print control character to text files,
** to handle these text files correctly the ASA/MCC bytes must be read as attribute from the delimited record
* Improve install.txt for z/OS (FLAMSTC + recommendation for LLA caching)
** STEPLIB I/O (3500 EXCPs) for the FL5CORE dynamic link library can be eliminated by LLA cashing
* If no record format is specified and length format is defined in record I/O, then default record format will be VAR
* Check for hardware implementation when using PGP with compression (better logging if ZEDC used)
* Support flat XML formats with only one root level in table support
** Prevent error "No initial root tag was defined (column definition without a root)"
** Don't print additional root elements ('table' and/or 'row') if root was given
* Support input output name mapping for file name in hash files
** Control over the file names of GNU and BSD formats
** Use the target file name at write as default
* Reduce usage of raw run (without pre-calculation) for character conversion to increase performance
* New FLAMSTC (FLZSTC) with ENQ protection of all resources and framework to run authorized functions (IFAUSAGE, SML-Logging, IDCAMS(GDG-Base), ...)
** At startup the FLAMSTC locks now all resources to prevent usage before anything is established
** All the users take care of these locks and are in wait state till anything is established
** No wait required anymore before the FLAMSTC can be used
* Reduce memory consumption in table support (compress and reorg structures on column level)
* Bugfixes
** Prevent segmentation fault (0C4) of subtask (FLZSRV) in FLUC subsystem if vdFcrPut failed (if open of file not possible) in vdFcrFlu
*** This issue will be recovered by FLAMSTC but the SYSLOG messages are not nice
** Prevent wrong instruction (0C1) in PCR routine of zIIP support based on missing resources if connect to WLM failed
*** Use ASCBJBNS/NI as WLM Subsystem Name
*** This issue is also recovered by the FLAMSTC it self
** Libssh was accidentally linked statically in version 5.1.24-29911 on z/OS
** Correct close of CLP in FLZSVR (vdClpChk() function) on z/OS (used for FLUC subsystem)
*** Depending of LE configuration the dynamic allocated memory could initialized
*** and in this case the offsets for the read/write and format strings are zero
*** this could result in an interface error at open of the FLUC record interface
** Ensure that FCRGTN/fcbgtn returns the table name defined by FCRSTN/fcbstn
*** The wrong table name was given after a FCRSTN followed by FCRGTN
** Improve handling of trailer data in ZIP archives
*** Support compressed data shorter then trailer length
*** Prevent error which request bigger block size
** Windows only: When opening a file via SSH without specifying a path (i.e. ssh://user@host/filename) the filename
was incorrectly converted to .\filename which would lead to a file not found error from the server
** Fix character to ASCII/EBCDIC conversion for non-Latin characters
** Don't remove whitespace after suppress padding in record I/O
*** Only use uiPadLen at read if uiPadLen is defined as parameter
*** Ensure as default suppression of space or binary zero
** Use known record length if text method REC is used at text formatting
** Suppress padding before writing the delimiter in record I/O
** Correct mapping of record format 'var_4b' of FLAM command for FLUC
*** Use length format 'HOST' instead of 'INTEGER'
** Take over known character set at record oriented write
** Correct handling of headlines (CSV) in table support
*** Don't read second line as head line if values identical with the headline
** Fix wrong handling of '-' for optional when reading in XML table support
** Fix handling of surrogates in UTF-16 encodings
- 0001015: [4. Element, Byte, Record or other API] Names space support for XML format in table support (Falk Reichbott) - resolved. - 0000999: [1.1 FLCL] Support SHA3 over CPACF and add support for RipeMD 256 and 320 in CryCore (Falk Reichbott) - resolved. - 0001021: [1.1 FLCL] Support open formats with ASA and MCC print control character to copy list and lmaps from USS to MVS (Falk Reichbott) - resolved. - 0001020: [1.1 FLCL] allocated space for FB files too small (Falk Reichbott) - resolved.
* Ensure dynamic link of libssh after change from SCLM to make file build on z/OS
5.1.24-29928
* Bugfixes
** Prevent segmentation fault (0C4) of subtask (FLZSRV) in FLUC subsystem if vdFcrPut failed (if open of file not possible) in vdFcrFlu
** Prevent wrong instruction (0C1) in PCR routine of zIIP support based on missing resources if connect to WLM failed
*** Use ASCBJBNS/NI as WLM Subsystem Name
5.1.24-29911
ATTENTION: Because of huge memory consumption the defaults object on
column level was removed from table support. Please define the
corresponding values directly for these columns.
* Support tag value delimiter format in table support
** The new general text format parser was built mainly to support SWIFT MT formats
** It can also be used to parse any kind of hierarchical text format including XML, JSON and others
** It can be used for a set of columns inside other column formats (partial parts with complex structure)
** This allows to parse fixed or variable length formats in between a piece of JSON or XML
* CPACF cryptographic acceleration for the 64 bit environment on Z systems is now supported
* Improved system information for hardware acceleration with CPACF on z/OS and USS
* Support ignoring a prefix in front of colon for comparison of XML tags in table support
** The same row specifications can be used if different name space abbreviations used
** The new parameter PREFIX in FORMAT.XML() (or XMLPFX in the default definitions) can be used
* Support "ROOT" as alias for "PATH" in table and row level
* Reduce memory consumption for header values at XML parsing in table support
** No growing of memory through collection of header values
** The memory usage is now constant independent of the amount of data converted
* Improve and correct reading of binary delimiter
** The record buffer is no longer used to store the delimiter (the full record length are used for record data)
** Implement special function for 1 and 2 byte delimiters to increase performance
* Add more PL1 samples (SPFCUCNV using FLUCUP, SPFLCICV using FLCICV (iconv like DLL of FLAM))
* Add FLAM4TDA as part of FLM4SEPA to the package
** Check feature code in assembler modules
* Initialize dia-critical characters support also if quiet is defined for the command line executor
* Improve performance in whitespace handling of ICV character conversion module
* Support enforcement of 32 or 64 bit ZIP archive format (prevent 7z warning)
** ARCHIVE.ZIP( ... FORMAT=Z32/Z64 ...)
** If no format is given the old behavior is used
** The default streamed format now is Z64
* Implement ICV character type check functions isICV - upper/lower/punct/space/cntrl/digit/xdigit
* Support 64 bit CCA support program with 64 bit integers on platforms where the C data type long has 64 bit
** The requirement to use 64 bit instead of 32 bit integers is determined by testing of the library
** This ensures that the SAPI library is handled correctly
* Reduce minimal keyword length for new FLAM4 executable to be more backward compatible with the old FLAM utility on Windows
** 100% backward compatibility is not possible, because of newly supported keywords
* Make implementation of new license modules compatible to old FLAM versions
** Using a new license module with an older FLAM revision could result in a segmentation fault
* Support enforcement of post-processing per file
** By default the post-processing of the I/O components was done only once after closing all files
** This resulted in only one post-processing call for the last file during directory walk
** With the new switch 'PERFILE' post-processing per file can requested
* Correct reset of environment variables (FL_PLATFORM) for Windows
** Use UNSETENV instead of SETENV with empty string if it was no variable defined before
* Correct mapping on Unix and Windows platforms in new FLAM4.exe and FLCL FLAM command if no record format is given and a record delimiter is specified
** Since 5.1.23 the record delimiter was only relevant if INRECFORMAT=TEXT/STREAM was specified
** Since 5.1.24 if no record format but a record delimiter was specified, then the record format will set to STREAM by default
* A new section in the FLAM4 manual and install.txt describes how to specify input and output record format as property
** Unify behavior of the new flam4.exe with the old one on Windows and Unix platforms
* Add symbol table and memory allocation statistics for CLP based on the environment variables below
** CLP_SYMTAB_STATISTICS - Print symbol table statistics to STDERR stream when closing
** CLP_MALLOC_STATISTICS - Print memory allocation statistics to STDERR stream when closing
* Add new parameter BLK2REC to WRITE.RECORD() and WRITE.FLAM4() of CONV command
** Add an additional delimiter based block to record conversion with suppression of trailing whitespace
** Required to convert for example a SWIFT-MT output format to a record per line
* Finalize each column after table change at write
* When using SSH configuration files with "IdentityFile" lines for public key authentication,
all matching keys are now attempted in the authentication phase.
** Previously, only the first encountered key was considered by libssh, all others were ignored.
* Make separator character in table definitions compatible between read and write
** when reading more than one separator character (SEPCHR) can be specified
** when writing only the first of them is used
* Introduce global literal caches per character set to reduce memory consumption
* Add log of license ID also at the end of the INFO function
* Bugfixes
** Fix several reallocation issues after enforcement of reallocation for internal write buffer
*** Normally the size of the write buffer is pre-calculated and only in rare cases a reallocation is required
*** Our test framework enforce now all this reallocations and in few components the handling was buggy
**** Fix possible damaged data after length error occur in table support
***** in very raw situations the internal buffer to write XML data in table support cloud be too small
***** in such case a reallocation of this buffer is done and the XML writer is called again
***** now we ensure, that the internal state of the XML writer is correct after an re-call with more memory
***** in the same rare case the offset for the element converter was not correct to write CSV format correctly
***** in the same rare case the input pinter for the formatting was not re-corrected after a re-allocation with pointer change
**** Fix wrong pre-calculation of required space for re-allocation in CNVBAS (prevent possible segmentation fault)
**** Fix segmentation fault after re-allocation with pointer change and correct amount of required space in FMTXML
**** Fix re-allocation issue in CNVPGP (the small sizes smaller than minimum space was not checked (the enforcement goes below (not really a bug)))
**** Verify output space before fractional length for big integer processing to ensure length error
**** Fix maximum length check in CSV-Support at write if masking requested
**** Correct re-allocation logic in FMTREC
** Fix whitespace handling "COLLAPSE" in ICV module (replacement was done with data instead of whitespace)
** Fix dirty delimiter handling of '0D..0DC285' in FMT.TXT()
*** The 0x85 was not removed from the data, which normally results in conversion errors
** Fix UTF-32 support in integer and float converters (plausibility check if length multiple of 4 failed)
** An issue in integer and float converters when using UTF-32 is fixed
- 0001017: [1.1 FLCL] ":32B:GBP31500," are a valid swift TAG with 31500,00 as amount (Tilo Eckert) - resolved. - 0001019: [6. Access Control, Logging and Errorhandling] Warning for soon to expire license is never shown to user (Falk Reichbott) - resolved. - 0000930: [1.2 FLCC] Support tag value delimiter (TVD (required for SWIFT)) formats in table support (Falk Reichbott) - resolved. - 0001009: [1.1 FLCL] Record length requires the delimiter length, but the delimiter is not part of the record (Falk Reichbott) - resolved.
* Support errors for missing fields when XML parsing in table support
5.1.23-28720
* Support for writing streamed ZIP files (i.e. no seeking to update ZIP headers)
** On z/OS, writing to a PS-VB dataset provides fast write speeds for streamed ZIP files
** Streaming is enabled via a parameter or by using STREAM as output file
*** Note: ZIP files written in stream mode cannot be read in stream mode and is therefore not supported
*** Reading the directory (located at the end of the file) is mandatory to extract streamed ZIP files
*** Regular (i.e. non streamed) ZIP files can be read in sequential form, but this is not yet supported
* Support large records (up to 1 MiB) for VBS datasets
** If no record length is given, the current maximum of 1 MiB is used as default
** Providing a lower maximum record length reduces the memory consumption for VBS processing
* FLAMFILES can now be read transparently from the default DD name FLAMIN and written to the default DD name FLAMOUT
* The JCL variable &FLTOKID can now be used to customize the token name of the started task FLAMSTC
* Control character in the range U0080-U009F are supported for UTF-8 if RPLCTR is enabled for text formatting
* Support silent and quiet as environment variables for CLE
* Support ALIAS, OBTAIN and TAPE by default for FLAM command in directory walk
** Ensures the same behavior as implemented by the old FLAM utility
* Too long records can now be skipped (new error handling mode)
* Return an error if memory allocation for first block fails for INFO GET.FILE
* Added more log messages for INFO GET.FILE
* Fixed a potential segmentation fault in interpretation of work area of the catalog search interface on z/OS
* Add first sample of FLUC record interface for PL1 (SPFCRGET)
* Correct code page for member name and comment of GZIP header when reading
- 0001012: [1.1 FLCL] FLAM command of FLCL needs ALIAS, TAPE and OBTAIN as default of DIR object (Falk Reichbott) - resolved. - 0001011: [1.1 FLCL] Support Unicode points 0080-009F at control character replacment in FMT.TXT (Falk Reichbott) - resolved. - 0000997: [1.1 FLCL] Supprt DEST=(dalsuser[,daluserid]) in FALLOC object for dynamic allocation (Falk Reichbott) - resolved. - 0000990: [1.2 FLCC] Add new column format RST to read the rest of an record or to a fix offset inside of an record (Falk Reichbott) - resolved.
* z/OS and z/USS code are now built on z/OSv2r4
** Minimum target z/OSv2r2
** Minimum architecture z12
** Tuned for z14
** Build Cobol samples with version 6.3 of the compiler
*** Requires toleration APAR PH15595 or rebuild of the COBOL samples with the SBUILD job.
* Support definition of padding character at read.record()
* Add FSTATZOS as object to z/OS package and adjust FLM4SEPA build job
* Support scan build for all z/OS sources
** Fix few scan build issues in FLM4SEPA source
* Use unique entry names for functions to prevent linkage problems if FLAMREC and FLCRECLB used in parallel
* Add more time stamps and CPU time measurement to printouts at command execution
* Add time stamp in front of each trace message to look at hangers
* Add a trace message at parsing for each find symbol
* Add new chapter to JCL considerations to prevent long runtime based on wrong STACK and HEAP definitions for LE
** For CEEOPTS we recommend to run FLAM on z/OS with:
*** STACK(1M,1M,ANYWHERE,KEEP,512K,128K)
*** HEAP(1M,1M,ANYWHERE,KEEP,8K,4K)
* Add CEE version to system info and check CEE version at beginning of FLCL against the required minimum on z/OS
* Add architecture level to system info and check the level at beginning of FLCL against the required minimum on z/OS
* Print uname(all) on POSIX systems at beginning of CleExecute to the log (Run on ...)
* Copy real data set name after FLMOPD into file name attribute at write of FLAMFILEs (replace DD:NAME by real name)
* Use always 'FLAMFILE' as default DD name in FIO.FL4() for FLAMFILEs if no DSN given
** Prefer a static allocation of DD name 'FLAMFILE' the default DSN (original name plus extension) given
** Uniform the behavior between FLCL FLAM and all other command of FLCL using FLAMFILEs
* Don't use simple default values in SVC99 (dynalloc) if data class given
** Only if the file size known, then the space parameter are defined
** DCB values are not set to defaults anymore if a data class given
** Since to this build a data class has no effect because all text units are defined with default values
* If available in catalog print data, storage and management class for files in info function on z/OS
* Support empty FLAM file name at read.flam4() in CONV command of FLCL ('DD:FLAMFILE' must be allocated in this case)
* Support static allocation of FLAMOUT at write.xxxx() if available before the default DSN is used on z/OS
** FLAMOUT works now like FLAMIN because it will be preferred if the allocation
** If the FLAMOUT allocation not defined and still no file specification given the default path/dataset name is used
* Don't determine member count at info function for empty file names anymore ('<SYSUID>.**' was the wrong default in this case )
* Static allocation of FLAMFILE are no longer freed in FIO.FL4() if no file specification given
* Fix sporadically errors in static allocation of DD names for old FLAM4 components
* All external links in HTML documentation are now open in a new tab and not into the frame
* Support DD names without 'DD:' as prefix (if only one qualifier and a static allocation available) on z/OS
* Use DD name SYSIN at read and SYSPRINT at write on z/OS if keyword STREAM used for FLAM4FILEs (not STDIN and STDOUT anymore)
* Fix 0C4 if no wildcard match if file lists used with old FLAM4 utility and subprogram
5.1.22-28493
* z/OS and z/USS code are now built on z/OSv2r4
** Minimum target z/OSv2r2
** Minimum architecture z12
** Tuned for z14
** Build Cobol samples with version 6.3 of the compiler
*** Requires toleration APAR PH15595 or rebuild of the COBOL samples with the SBUILD job.
* Support definition of padding character at read.record()
* Add FSTATZOS as object to z/OS package and adjust FLM4SEPA build job
* Support scan build for all z/OS sources
** Fix few scan build issues in FLM4SEPA source
* Use unique entry names for functions to prevent linkage problems if FLAMREC and FLCRECLB used in parallel
* Add more time stamps and CPU time measurement to printouts at command execution
* Add time stamp in front of each trace message to look at hangers
* Add a trace message at parsing for each find symbol
* Add new chapter to JCL considerations to prevent long runtime based on wrong STACK and HEAP definitions for LE
** For CEEOPTS we recommend to run FLAM on z/OS with:
*** STACK(1M,1M,ANYWHERE,KEEP,512K,128K)
*** HEAP(1M,1M,ANYWHERE,KEEP,8K,4K)
* Add CEE version to system info and check CEE version at beginning of FLCL against the required minimum on z/OS
* Add architecture level to system info and check the level at beginning of FLCL against the required minimum on z/OS
* Print uname(all) on POSIX systems at beginning of CleExecute to the log (Run on ...)
* Copy real data set name after FLMOPD into file name attribute at write of FLAMFILEs (replace DD:NAME by real name)
* Use always 'FLAMFILE' as default DD name in FIO.FL4() for FLAMFILEs if no DSN given
** Prefer a static allocation of DD name 'FLAMFILE' the default DSN (original name plus extension) given
** Uniform the behavior between FLCL FLAM and all other command of FLCL using FLAMFILEs
* Don't use simple default values in SVC99 (dynalloc) if data class given
** Only if the file size known, then the space parameter are defined
** DCB values are not set to defaults anymore if a data class given
** Since to this build a data class has no effect because all text units are defined with default values
* If available in catalog print data, storage and management class for files in info function on z/OS
* Support empty FLAM file name at read.flam4() in CONV command of FLCL ('DD:FLAMFILE' must be allocated in this case)
* Support static allocation of FLAMOUT at write.xxxx() if available before the default DSN is used on z/OS
** FLAMOUT works now like FLAMIN because it will be preferred if the allocation
** If the FLAMOUT allocation not defined and still no file specification given the default path/dataset name is used
* Don't determine member count at info function for empty file names anymore ('<SYSUID>.**' was the wrong default in this case )
* Static allocation of FLAMFILE are no longer freed in FIO.FL4() if no file specification given
* Fix sporadically errors in static allocation of DD names for old FLAM4 components
5.1.22-28284
* Fix error at write of FLAMFILEs if static allocation with less than 512 byte record length done using the old FLAM utility on z/OS
** The problem was introduce with the first build of version 5.1.22 of FLAM with a fix of another problem (for FLAM-VSAM record interface)
* Fix wrong mapping of encoding strings to CCSID for code pages 'EUC-JP/TW/KR/CN'
* Remove remaining dependencies from FLM4SEPA modules to FL5CORE (getFileSize, remove_hfq)
* Remove remaining dependencies from FLAMREC modules to FL5CORE (FSTATZOS, CHKMEM, CLEPUTL, FLAMUTL, ENVUTL, DYL and STR)
5.1.22-28224
* Support new format 'RST' in table support to parse or write data up to a certain offset or end of the record
** Some time in variable length records the next variable part starts at a certain offset
*** This component support to read the remaining rest of the data as column up to this offset
** Some time it is useful to read the rest of an record in a column
*** This component supports this by default (offset=0) to read the remaining rest of the data
** At write such space must normally filled with padding bytes and can be added up to the defined offset
* Support 1 to 10 decimal digits as length filed in table support at read and write
** e.g. COLUMN(FORMAT.LFD(LENFMT=D5X)) for a 5 byte decimal digit length field not including it self
** often used for record length or parts of records in COBOL copy books
* Fix wrong adjustment of format parameter in table support
** Prevent setting of format.fix() values if other format (not the default) used
* Correct handling of MAXLEN parameter in table support
** If default format is FIX an it includes a variable length fields, then no padding to the defined MAX/FIXLEN should be done
* Few improvements and corrections of FLM4SEPA
** Fix wrong handling of BTAG in FLM4SEPA
** Add CPU time measurement in FLM4SEPA
** Rewrite FLM4SEPA from bytes wise to an optimized 32 bit 4 times parallel handling
** Check length fields of load records
** Improve statistics and log output
** Don't read next record if Bloom count achieved
*** Prevent decompression of next matrix to improve performance and reduce CPU consumption
* Support destination user and id and redirection to hold queue at SYSOUT allocation
* Print data position inside the block or record to error messages in table support
* Correct allocation of extended attribute datasets (text unit with wrong pointer)
* Binary zero (0x00) is now whitespace at character conversion (like in all other cases)
* Support concatenation of member in FIO.FL4() component if member name has not changed
5.1.22-27996
* Support padding bytes at the end of a FLAMFILE on Windows and UNIX platforms (incl. USS)
* Correct INFO GET.FILE if seek for ZIP files result in an error (e.g. if VSAM dataset on z/OS)
* Correct resource file for Windows packaging of flam4.exe
* Add FRCGFN to MF-EDZ support
* Add new feature FLM4SEPA
5.1.22-27795
* Support double KeyID's at import of PGP keys
** Allow the same key value for top and subkeys (is a security issue)
** Support a RACF policy (KEY.IMPORT.PGP.DOUBLE.KIDS) to prevent this
* Support a new environment variable FL_PGP_PREF_SYMALG_AES128
** Can be set to YES to invert the order of preferred symmetric algorithms for AES
** Change order from AES256/192/128 to AES128/192/256 for all FKM5 modules at generate and export
* Correct Windows and Unix packaging (replacement of the FLAM4 executable)
5.1.22-27678
* Improved documentation
** New CLEP with better documentation generation
** New built-in function HTMLDOC
* SSH improvements
** An undocumented new libssh feature automatically loads OpenSSH per-user (~/.ssh/config) and system-wide (/etc/ssh/ssh_config) config files
*** This introduced a regression in version 5.1.19-22166 which could override user-provided SSH options with options from a config file
*** This release restores the intended behavior of user-provided configuration options taking top priority
** The net.ssh() object has several new config options:
*** CONFIGFILE=<configpath> (allows to specify a custom SSH config file in OpenSSH format)
*** NODEFAULTCONFIG (disables loading of per-user and system-wide OpenSSH config files)
*** ALGORITHMS() (object to specify custom lists of acceptable / preferred crypto algorithms)
** SSH configuration options are priorized in the following order:
*** 1. Command string options
*** 2. Custom configuration file (via net.ssh(configfile=...))
*** 3. Per-user configuration file (~/.ssh/config)
*** 4. System-wide configuration file (/etc/ssh/ssh_config)
*** Example: Path to known_hosts file is set via net.ssh(knownhosts=...) and in a config file: value from net.ssh() is used
** If CONDBG logging is enabled, the effectively used username, host, port and known_hosts files are logged (after processing all config files and parameters)
* Support SYSOUT allocation to write files as mail attachment on z/OS
** write.text(...falloc(sysout(format.mail(from='sender@mail.com' to='receiver@mail.com'))...)
* Improve separation of binary data from base encoded data at detection
** If no ARMOR header/trailer found (simple base encoding) then character set detection is now done
** If the character set detection must determine UTF-8, ASCII or EBCDIC in each other case (binary data) this could no base encoding
* Support comment (user header) also if no system header written with old FLAM4 utility and subprogram
** If IDEVICE=USER used then no comment war stored in the member header
* Support error trace in I/O components (for better and simple problem determination)
** Support better reason codes (No such file or directory, permission denied, ...) if file I/O failed
* Correct lexem to lexeme or lexemes in each case (CLEP change)
** the Built-in function "LEXEM" are still available as abbreviation/alias of LEXEMES
* Support large files in FLAM4 components at dynamic allocation on z/OS
* Support extended attribute datasets in FLAM5 components for dynamic allocation on z/OS
* Improve support of empty records in variable length record datasets on z/OS
** If environment variable '_EDC_ZERO_RECLEN' set to Y (default) then records of length 1
with a padding byte (0x00 or 0x40) as content will be records of length 0 at read
* Support new environment variable 'FL_INFO_BLOCK_SIZE' to setup another block size for INFO command
** This is mainly useful for PGP files to get the PGP dump
* Support new special condition code 160 (FLMRTC_NBS) which tells that a larger block size is required
** This could happen for example if PGP use BZIP2 compression with large (900KiB) blocks, in this case 1MiB would be a useful block size.
* Improve error handling in I/O routines on z/OS
** Call I/O error handler if system call fails to write _amrc structure into the log
* Print external file attributes in member list of ZIP archives
* Add new switch 'NOXFLD' for ZIP archives to prevent write of extra fields to member header and directory entry
* Support new entry FCRGFN() and fcbgfn() to determine the file name after open and before close
* Add time stamps to CLE messages and overall run and CPU time at end of execution
** Useful to see run and CPU time of CLEP usage around FLUC, FLAM or FLIES subprograms
* Improve parsing of URLs (more fail save)
* Improve IP communication for ClamAV
* Reduce default record for long records to limit the buffer size to a maximum of 256KiB plus 1 record
* Bugfixes:
** Don't return end of member (6) in FLMGKY anymore
*** A additional call again FLMGKY to handle this error was failing always with key not found (5)
*** Now FLMGKY handles EOM internally an return the next matching record in the next member
** z/OS/USS only: Fix libssh signature verification at re-keying after a certain time (1h) or amount of data (1GiB)
** Correct usage of UserIO in FLAM4 components (now works as specified)
** Fix space allocation parameter handling in FLAM4 components on z/OS
*** Correct large file handling in dynamic allocation for FLAM4 components on z/OS
** INFO function don't result in damaged FLAMFILE for VSAM-KSDS-FLAMFILE if FLMPOS failed with missing key in KSDS
** Don't accept block mode = 0 in UNIX/Windows source of FLAM4 components to prevent write of non-readable FLAMFILEs
** Use determined record size in FLMOPD in FLMOPF if FLAMFILE open for update (only FLAM4 on z/OS)
** Return maximal record count for the matrix at read of a FLAMFILE (only FLAM4 on z/OS)
** Set maximal record count at read dependent on the determined compression mode (only FLAM4 on z/OS)
*** If the provided MAXREC parameter greater then 255 or 4095 then the maximum will be used
*** No error anymore if MAXREC greater than 4095
** Correct comment handling between CNV.GZP and ARC.ZIP
*** ZIP file member comment can now successful disables
** Fix FLGT error with additional new lines if FLGT used to edit a file
*** The file is read in junks and after each junk a newline was added with appending the junk to the text of the editor
*** Now the whole file is read into the memory and the editor text is set without changes
** FLGT now requires the user to confirm a requested save action when the filename was changed and a file with this name already exists
** FLGT checksum output does not swallow leading zeros anymore
** FLGT does properly remove temporary files, but not when external program was used to view it or an error occured
** Set key position and key length to 0 for non KSDS data sets at write if no key disposition defined
** Don't use national (dia-critical) characters in CLISTs anymore to prevent code page issues
** Use connection to remote FLAMFILEs to create directories and to verify of file already exists
** Link against IBM zlib version 1.2.11 to support ZEDC CPACF command on z15
*** Prevent empty CRC32 in GZIP, PGP and ZIP files and the corresponding format issues on other platforms
- 0001008: [1.0 FLAM4 or older] FLAM 4 utility produce 0C4 if file list don't match (Falk Reichbott) - resolved. - 0000996: [1.1 FLCL] Support digit based length fields at FORMAT.LDF (Falk Reichbott) - resolved. - 0000991: [2.1 Subprogram FLAM (COMP/DECO)] FLAM4 for Unix/Windows don't support padding bytes (Falk Reichbott) - resolved. - 0000987: [c. Deployment] Keep the last stable revision / release for download (Ulrich Schwab) - resolved. - 0000983: [4. Element, Byte, Record or other API] Support FLUCUP-Interface (including old FLAMUP) on Java (Tilo Eckert) - resolved. - 0000835: [2.2 Subprogram FLUC (CONV)] Support write of files as mail attachment over the SYSOUT(TCPCSMTP) (Falk Reichbott) - resolved. - 0000989: [4. Element, Byte, Record or other API] New entry to get the file name at write (Falk Reichbott) - resolved.
* Fix charset issue when appending a new key to known_hosts on EBCDIC systems
** Key type was written in ASCII
* Correct read of key entries in known host after damaged or unsupported entries
* Reduce stack size for zEDC-Support from 128 to 64KiB on z/OS
* Fix error at delete of PGP keys for PKCS#11 devices
5.1.21-26329
* Print SAF control information in system info (flcl info get.system)
* Write records to stderr and stdout files in command processing if text
** Will also result in CRLF(0D0A) instead of LF(0A) as delimiter on Windows
* Write of blocks grater then 256KiB per SSH
** No "remote channel is closed" error anymore in such case
* Support complex PGP certificates at import
** Containing Notation Data, Embedded Signature and other non supported certificate content
** Use a blackbox to passthrough such content if a certificate store used
* Support mapping of codepoints to nothing through user table (*0D=)
* Add load module FCURSN to z/OS installation package
* Fix wrong handling of DD and pathnames at write of simple files on z/OS
* Add FLM4UP load module and FLM4UPLB DLL to z/OS package
** Remove wrong DLL FLAM4UP from load on z/OS
* Add main HTML page for whole documentation on Unix an Windows
* Correct and improve IO-ABEND handling on z/OS
** If VSAM record not found or EOF reached the VSAM I/O-ABEND is ignored also if fill bytes not 0x00
* Use relative path for SAF check if absolute path not available
5.1.21-26124
* Improved performance in main code paths (FIO, CNV and FMT components) due to a reduced call stack
* Byte interface function fcbnewmbr() is now avaible as method newMember() in Java Byte Interface
* A new example in the Java byte interface JAR file demonstrates how to call FLCL from Java
** Includes an example for launching the native z/OS variant of FLCL using the TSO command on USS
* Property files containing unknown property names are no longer rejected in FLM4UP/FLAM4 utility
* CX8/VR8 encryption support in FIO.FL4 component
* Default physical block size for non-blocked record formats is now the record length (block=record)
* Improved record handling automatism for FLAMFILEs
* Improved automatism in record handling if control character replacement is used
** Switch from binary to text processing
** Switch from FBA/M VBA/M to FB/VB record format
** Adjust all corresponding data attributes
** The correct padding character is used if the character set is known
* If a record-oriented file with print control characters is written
and a host record format without print control characters is defined (e.g. FB/VB)
and the print control is retained then the ASA or MCC bit is activated (e.g. FBA/VBA)
for the defined host record format
* Added support for CHRMODE and SKIPEQUAL parameters in read.char/text/record/flam4/auto()
** SKIPEQUAL switch prevents UTF-8 to UTF-8 conversions (i.e. invalid characters are not detected)
* SAF checking of URLs is done fully qualified
* Bugfixes:
** Insert handling for EXK10 user exit in FL5 components fixed (FIO.REC())
** Fixed uninitialized function code in old FLAMUP for EXK/D10 user exits
** Fixed wrong behavior if GDG(+1) used to create a new generation (don't create PO anymore)
*** Support for (+0) and (-0) as valid GDG (same as (0))
*** PO-GDGs are only supported if organization PDS or LIB is set
** Fixed OUTPATH usage (correct concatenation of output path and filename)
** Allocation error fixed that was introduced in previous build (Support FIOURL SAF request also for all help files (reports, command outputs, ...))
*** Help output data set names are allocated relative prefixed with an additional SYSUID
5.1.21-25699
* Improved local pre and post command execution
** Uses OS-specific implementations for improved performance and process control
** Execution time can be limited by setting a timeout (except for z/OS)
*** On timeout, the process and its children are killed (except for z/OS and SSH)
* Add FLM4UP documentation to z/OS package
* Use 'type=memory(hiperspace)' for temporary file on z/OS by default
* Support FIOURL SAF request also for all help files (reports, command outputs, ...)
* Correct handling of remaining rest of output in command processing
* Correct new line handling and support '\r' (old Mac) if command output written to log
* Put connection in front of remote commands for SAF resource
** Gives control about local and remote commands
** and where the remote command can be executed
* Support EBCDIC new line to line feed handling for internal string conversions
* Support identity mapping as error handling at character conversion in each case
* Add transliteration as error handling option (abbreviation for MODE=IGNORE SYSTAB=ICONV)
* Add character conversion mode parameter to command pre- and post-processing (CHRMODE)
** Gives control about error handling for damaged or non convertible characters
** Ensures character conversion (character conversion if CCSID are equal)
* Fix dynamic allocation if multiple member (a lot of files from USS) written to a less amount of PDS(E)'s (on ZOS)
** Correct handling of RENEW parameter in this case also (only delete the PO if it first time written)
* Print member count of a PDS(E) in info list and correct file organization (library for PDSE instead of PDS)
* Correct dump processing for memory and temporary files (add always a new line like it was done for streams)
* Fix memory leak in FLAM4 component FLMFNUC if MODE=NDC was used
* Support EXD10/K10/D20/K20 exists in FLAM command of FLCL
* Add FL4.CONTROL switch for EXD10/K10/D20/K20 and USERIO to SAF support
* Fix recovery loop if enclave token became invalid touring mass parallel zIIP SRB processing
* Correct and review documentation (mainly for command processing)
ATTENTION: With version 5.1.21 list of files are supported as input. To
be backwards-compatible with FLAM4, these files can be comma-separated
from each other. This requires support for arrays after a simple assignment.
(file=text.txt,binary.bin).
A bug was found, which allows the assignment of a keyword to a variable
with the same name (name=name). In this case the keyword name was
interpreted as string 'name'. However, name is a keyword for a undefined
variable in this case and a corresponding error is logged. To
prevent this error you must enclose the string in quotes (name='name').
Additionally, a separation of the begin of an explicit array definition
using '[' from an index used to reference a certain value from an array
was required. To provide an index in an expression, curly brackets '{}'
must now be used (newword=array[1] must be changed to newword=array{1}).
The index must be a number. Expressions are no longer supported.
5.1.21-25177
* Add support for pre- and post-processing of commands when reading read or writing files
** Can be used to prepare something in front of an read or write request
** Can also be used to cleanup or issue a receipt after successful write or read of files
* Support SAF protection for feature codes, other resources and several policies (see install.txt for z/OS)
** To control parameter strings (CLEP)
** To secure PGP key management
** To secure SSH connections
** To secure pre- and post-processing commands
** To secure all service provider interfaces (FAVE, FCPE, FKME)
** Control access to PGP user and key IDs
** Control access to complete URLs containing connection, path, filename and member information
* Add support for list of files at read (file='name.ext','dat*.txt' or file=>'DD:LIST')
** Support parameter files for simple arguments (keyword=>filename)
** Support arrays of simple values after assignment (keyword=hugo,berta detlef)
** Improve error handling of FLCL utility and FLUCUP
*** In the case that the amount of failed operations is identical with the+
amount of files processed the completion code is 8 and not 4 if more+
then one file processed
*** In simple words, if any file fails 8 will be returned
*** If one file succeeded and one of the other fails 4 will be returned
* Add zIIP support for MODE=ADC in FLAM4 components
** Only if zIIP not available zEDC-Support will be tried like in the past
* New FLCLBOOK with reduces memory consumption and better usability as DOCZIP
* Correct support for user header (comment) if CX7 used in FL5, FL4U and FL4W projects
* Support compression limit in FIO.FL4() and write.flam() objects
* Return an error if FKME or passphrase provided but the FLAMFILE is not encrypted
* Print FKM5 error message in each case (was not printed if key was not found)
* Support appending of FLAMFILEs on z/OS (OPENMODE=4, will fail in Windows an UNIX)
* Improve error handling in FIO.FL4() if DMS error occurred
* Add new ISPF lines command FLDEL to delete files
* Support logging and tracing if remote (SSH) access to FLAMFILEs are used
** Missing entries for example (remove of FLAMFILE) are now visible
* Reduce requirement to use && to escape mapping sequences for dia-critical characters
* Don't print obsolete license information in FLAM4 for Windows anymore
* Support writing to DUMMY for the output file in UTIL command
* Add usage log status to system information (flcl info get.system)
* Increase the size of internal error buffers and error trace by factor 2
* To avoid ABENDs be default we change the logic for system variable &FLREUCEE
** To activate reusing the language environment the system variable must be defined &FLREUCEE=ON
** In all other cases the performance feature is disabled
* Don't include files in directory walk on z/OS where the SVC27 (obtain) failed
** If the warning for such files needed then the flag OBTAIN in the DIR object must be defined
** If the VOLSER equals 'MIGRAT' or 'ARCIVE' and SVC27 failed then the file is marked as on tape
* Support lower case (a-f) in hex strings for FLAM4 components on z/OS
* Don't get and set file attributes for streams (reduce amount of warnings)
* Bugfixes
** Some corrections of old FLAM4 components
*** Correct error handling
*** Prevent ABENDS in error situations
*** Creation of RRDS and LDS VSAM datasets
*** Fixed segmentation fault if FKMEFILE if comment used
*** Correction for 0C4 in FLAM when started under ISPF
*** Fixed memory error in VSAM-RRDS and RECSIZE=0
** Fixed wrong statistics in FLAM4 archives summary
** Fixed destruction of stack if records extended in length if key disposition new used
** Fixed use after free in close function of FIO.FL4() if directory walk used
** Fixed segmentation fault after switch of the command in FLCC (re-initialization issue)
** Fixed memory exhausting issue in character conversion module
*** If the same conversion reused in directory walk for the next file, the maximal expansion was used 255
*** Now a warning is written to the log (completion code 1) in this case and the case should not happen anymore
** Fixed mistake in usage log structure introduce with 5.1.20-23920
*** Usage log was written, but processed amount of byte was written in the wrong place
*** The result of this error was that the usage log records written are not processed for usage log summary
** Fixed wrong file attributes used in FLAMUP if system header not created on z/OS
*** Could result in large file allocation or some other errors or wrong behavior
- 0000986: [1.2 FLCC] Add switch SKIPEQUAL to read text (Falk Reichbott) - resolved. - 0000981: [1.1 FLCL] Migrated files result in warning and completion code 4 on z/OS (Falk Reichbott) - resolved. - 0000958: [6. Access Control, Logging and Errorhandling] Error trace lost information if table output needs the whole space (Falk Reichbott) - resolved. - 0000977: [f. Documentation] Add cover, main and over missing pages to FLCC Help (Ulrich Schwab) - resolved. - 0000901: [2.2 Subprogram FLUC (CONV)] Add pre- and postprocessing script execution in SSH support (Tilo Eckert) - resolved. - 0000976: [1.1 FLCL] Support 3072 bit RSA keys as ModLen in PGP (Falk Reichbott) - resolved. - 0000879: [2.2 Subprogram FLUC (CONV)] Support SAF request in SW implementations for key access and other resources (Falk Reichbott) - resolved. - 0000880: [2.2 Subprogram FLUC (CONV)] Support encryption and key management policies (Falk Reichbott) - resolved. - 0000820: [z. Other] FL4U: Printout of license id cannot be disabled with show=none and wrong old license is logged (Ulrich Schwab) - resolved.
* Add string search feature to FLEDIT Gui-Tool (FLGT)
* Fix 0C4 if FKMEFILE used with wrong DD name on zOS
* Support new format parameter for ARMOR base encoding to define which key value pairs are printed (OFF/STD/EXT)
5.1.20-23920
* Support for print control character replacement when reading
* Improved error handling in zIIP support (only if ZIIP-Feature available)
** Provide correct WTO in each case
** Don't issue a WTO for consequential errors
* Use owner ID also at export if a user ID is added and no private key available
* Features can be deacticated through system symbols (z/OS) or environment variables (all other platforms)
* Add missing break in fcbread (fix wrong EOT error instead of EOF on API level)
* Fix wrong comment and file name in state string after read of GZIP member on z/OS (IBM zlib has different behavior)
* Fix MCC replacement at write (wrong value in switch case prevent correct replacement)
* Few improvements for FLAM GUI-Tools
** Fix FLGT registry entries and empty info page
** use monospace font for info and checksum on all platforms
** Support different fonts for edit and save the selected font permanently
** Set command to checksum if algo option is used
** Add list option to print available checksum algorithms
** Add menu option to toggle line number display
* Write of KSDS with key disposition delete makes no sense - change key disposition in this case to 'OLD'
* Hide FKME parameter (could include credentials) and make FKME parameter binary to support C'', A'', E'', X''
* The JNI wrapper DLL (FLCBYTJV) for Java byte interface is now part of the z/OS deployment
** Add load module (DLL) and import file for linking to z/OS XMIT and SMPE package
** Add sample job FLAM.JOBLIB(JZOSFCBI) for use of Java byte interface with JZOS wrapper
** Support "FLCBYTJV" as additional default library name in ByteInterface.jar (simple STEPLIB concatenation possible)
*** Provide new JAR file in ADDON download area (https://flam.de/de/download/addons/Java/ByteInterface/)
* Add new CLIST FLVEDIT1/2/4/8 with limited record length and no cut of longer records
* Correct calling conventions for old assembler modules without a stack frame
* Add OUT parameter to UTIL command to allow write of printouts to certain files
* Don't overwrite record length in FMT.REC() with origin if record length is set from outside
* Rework FKMEFILE to a RENT module to prevent ABEND if authorized load is used
5.1.20-23498
* Few PGP enhancements to import of keys and certificates
** Support signature directly on a key (required to import keys from z/OS EF)
** Support persona and casual certificates like positive and generic certificates
** Support primary key binding signature like subkey binding signature
** Support new user IDs for public keys at import by signing with the owners private key
** Improve error messages
* Improve file organisation and record format handling
** Adjust file organization to a VSAM one is the record format defines a VSAM one
** Use standard record (VE/VK/FR) format if file organization changed to ESDS/KSDS/RRDS
** Calculate record length in each case correctly (also if PRNCTR=RETAIN defined)
* Support print control character replacement in FIO.REC() at write
** Print preparation can be activated if a ASA or MCC dataset written to a non ASA MCC dataset
** The print control character (MCC or ASA) are replaced by additional records so that the written file looks like printed
** Default lines per page is 60 but can also be free defined
** Correct ASA/MCC mapping and support all channels as a kind of form feed
** Change default MCC byte from 0x03 (no operation) to 0x09 (write and space 1 line )
* Correct and improve key disposition handling
** Support KEYDISP=NEW with records shorter then KEYPOS
** Add KEYLEN to default RECLEN if KEYDISP=NEW
** Set default KEYPOS and KEYLEN if KEYDISP=NEW/DEL
* Fix handling of record delimiter in FIO.FL4()
* Add missing free in FCRBFIN (not all allocated memory was freed)
5.1.20-23372
* Support for re-keying of PGP files (very usefull in conjunction with flucFS views)
** Add, set and replace recipients in an existing PGP file
** Only re-encrypts the session key without re-encrypting the data
** Always skip session key packets in PGP files if FKM5 call failed
*** If passphrase is used then no FKM5 object must be given
* Support for read-only views in flucFS
** Allows different logical views to the same physical data
** PGP files encrypted with a repository key can be made accessible as plaintext (e.g. for employees) and encrypted files (for external partners) at the same time via different views
** Some corrections in flucFSConfig GUI tool
* FLCC and flucFS configuration support parameter files
** Support object and overlay assignment to a parameter filename in FLCC
** Parameter trees can be exported as parameter files using FLCC
** Parameter files can be imported into the parameter tree of FLCC
** Main keywords are used instead of aliases in parsed parameter list output
* Unicode 12.0.0 is now used for validation of codepoints
** Reserved codepoints are also accepted (i.e. are considered valid codepoints)
* SSH session plaintext packet capture to a file can be enabled in net.ssh() object
* FLAM GUI Tool (FLGT) improvements
** Integration of FLCHECKSUM in GUI tool
** New personalized windows
* Improved handling of default parameters in table support
** A default value (CCSID) does not override a known value for a column
* Improved alignment in CSV module of table support
** The last element per line is also aligned
** Alignment requires enclosing characters
** Improved performance
* Support writing of members to the same open archive on stream, byte and record interfaces
** Significant performance improvement compares to opening the archive for each member in append mode
** New function fcbnewmbr()/FCRNEWM() which implements re-open for the new member
* Support hash and MAC calculations (including HMAC) for the in-memory interfaces (fcbhash/hini/hadd/hfin and FCRHASH/HINI/HADD/HFIN)
** A job example and program in Cobol can be found as SOFCRHSH in JOBLIB and SCRLIB
* A set of functions for in-memory base en/decoding was added (fcbbase/bini/brun/bfin and FCRBASE/BINI/BRUN/BFIN)
* New function to query statistics information (fcbcount(), FCRCNT())
** Useful to know how many bytes or units are processed after formatting or before I/O
* Support NOARCH switch to disable the attempt to read archives automatically with CONV command
** If the file is known to not be an archive, this prevents repeated attempts of opening a file
* New way to query system information (flcl info get.system)
* A warning is now printed if a given system, local or default ASCII/EBCDIC CCSID does not fit with the detected character set
* Log output now contains always the license number
* Error messages produced by utilities are prefixed to know which utility was executed
* Improve DYNALLOC error handling and check key offset and length for VSAM-KSDS
* Mofifications of the FLCL FLAM command to match behavior of the FLAM4 utility on z/OS
** New parameter NORUN in CONV and XCNV command (acts as SHOW=ATTRIBUTE in old FLAM4 utility)
** Empty string on the right side of assignments is now allowed (e.g. "comment=" no longer results in a syntax error)
** Empty comments in FLAMFILEs are allowed
** Logging types "SOURCE" and "TARGET" added for printing the actual name used when reading/writing (required for SHOW=DIR)
** Ensure FB-512 as default for FLAMFILES (was FIX-512)
** New special condition code 80 (compression limit was not achieved) // "Kompressionslimit wurde nicht erzielt" Was soll das heißen? (PS: Du benutzt achieved fast immer falsch. "to achieve" = "etwas bewirken / erreichen")
** New special condition code 88 (read file is not a FLAMFILE or corrupted)
** Support DD:FLAMPAR on z/OS (used if only a command without assignment or parameter is specified)
** Support DD:FLAMFILE/FLAMIN/FLAMOUT/FLPRINT
** Required strings are only terminated by separation characters (space or comma), comments or closing brackets on level 0
*** i.e. PDS member string assignment like FILE=FLQ.PDS(MEMBER) without quotation marks is possible
* Add FLMSIGA/N modules to FLAM.OBJ (FLAM search and find solution for TDA)
* Bugfixes
** FLAM4 components on z/OS, did not recognize some I/O errors currently when reading a USS file
*** ABEND was triggered if opening a FLAMFILE in USS failed
*** ABEND was triggered during directory walk if FLCL CONV with path names was used
** Initialize system header length when reading and allow to read system headers of each platform
*** If a member was read from a FLAMFILE which was generated with FLAM5, the system header was not evaluated on z/OS
*** As a result, the recreation of VSAM datasets was not possible
** Fix of error handling if FIXLEN formats are used
*** If output data does not fit within a fixed-length field, error code FLMRTC_LNG is returned instead of FLMRTC_LEN
** Correct error handling in close function in block-oriented I/O using records
** Don't restrict masking processor to strings type (only the character set must be set)
** Fix for local character set handling in character conversion module
*** Always use the local CCSID and do not override it with the detection results
** Warnings are logged if error logging is enabled
** Error trace is logged if minimal logging is enabled
** Improve printouts of operating system of ZIP, GZIP headers
** Fix external attribute handling when writing ZIP members (set correct file permission bits for files in archive)
** Correct default POSIX permission bits from "rw- r-- r--" to "rw- r-- ---"
** Fix FKMEFILE for Unix and Windows systems (E' or A' or C' as end of a key in hex notation resulted in an empty key)
** Correct linkage of FKMEFILE for z/OS (prevent 0C4/04 if FLAM load APF authorized)
** Endless loop in FLAM4 for Unix and Windows if password length was 0
** Fix statistics error in table support (some data was counting twice)
** Make CLIST FLVEDIT on z/OS abend-save
- 0000970: [1.1 FLCL] Add possibility to overwrite all armor key value pairs (Falk Reichbott) - resolved. - 0000968: [b. Packaging] Add JNI for JavaByteInterface to z/OS package (Tilo Eckert) - resolved. - 0000965: [1.1 FLCL] Fix font in FLGT (GUI tool) editor and view (Ulrich Schwab) - resolved. - 0000964: [4. Element, Byte, Record or other API] Add function for base encoding to byte and record interfaces (Falk Reichbott) - resolved. - 0000886: [1.1 FLCL] New info function for system evaluation (Falk Reichbott) - resolved. - 0000959: [1.1 FLCL] Multiple reads of CONV command result in timeout issues, because file is moved after close (Falk Reichbott) - resolved. - 0000961: [4. Element, Byte, Record or other API] Support HMAC and HASH calculation on API level as memory to memory interface (Falk Reichbott) - resolved. - 0000963: [4. Element, Byte, Record or other API] Control a maximum limit of written bytes to prevent to large files (Falk Reichbott) - resolved. - 0000842: [4. Element, Byte, Record or other API] Write of more then one member to a ZIP/FLAM archive with FLUC interface (Falk Reichbott) - resolved. - 0000749: [1.2 FLCC] Support string type file and parameter file management with FLCC commander (Ulrich Schwab) - resolved. - 0000925: [1.1 FLCL] Add or replace recipient or re-keying an existing encrypted PGP file (Falk Reichbott) - resolved. - 0000948: [3. Subsystem, Driver, Plugin, SPI] Add support for different logical views on the same physiscal folder in flucFS (Ulrich Schwab) - resolved. - 0000812: [2.2 Subprogram FLUC (CONV)] Use zIPP capable SRBs for RUN functions of FLUC and for segment serialisation and de-serialisation in FLAM4/5 kernel on z/OS (Rolf-Dieter Euler) - resolved.
* Fix FLAM record interface stubs for Micro Focus EDZ support (introduced with SSH support for FLAMFILEs)
** Add new target libfl4recuc.dll/so to provide the FLMXXX functions in upper case
** Without the libfl4recuc, the function names are in lower case prefixed with '_' and appended with '1' (_flmxxx1)
* Set always record format, record length and block size in inverse command of FIOREC
* Improve error messages of FL4REC (separation between DMS and FLAM internal error)
* Support escape sequences for special characters and code pages for all strings, file names and labels
* Correct and check file pointer for print outs concerning quiet and silence switches of FLCL
* Check feature code for CNVAVS (anti virus scanning)
* Avoid conflict with flamdoc package for several Linux distributions
* Improved version of FLEDIT (flgt -c edit filename) GUI tool
5.1.19-22166
ATTENTION: Change of default CCSIDs and environment variable handling for APIs
Only relevant if no LANG variable is set:
To fix inconsistency problems concerning different default CCSIDs in FL5
and the command line parser CLEP, default CCSIDs have been unified. On
ASCII-based systems, ISO-8859-1 (Latin-1) is the default. It EBCDIC
systems this is IBM-1047 (Open Systems Latin-1).
A system not using IBM-1047 (e.g. IBM-1141 for a German host), the
environment variable LANG must be set appropriately to avoid charset
issues (see install.txt for z/OS). Additional escape sequences, CCSID
areas in commend strings and a new environment variable for CCSID of the
CLP string are now supported. The latter two features are designed to
support literals with inconsistent CCSIDs, if FLAM APIs are used with
CLP strings.
The subprogram, byte, stream and record APIs up to version 5.1.18 loaded
system variables and standard environment variables as part of the API
calls. This prevents control about the environment by the application
programmer. Starting with FLAM 5.1.19, API functions no longer load any
system or environment variables, but provide new functions (fcbenv,
FCRENV, flucenv and FCUENV) for it. These functions allow application
programmer to set those variables as needed.
* Add FLUC subsystem for z/OS (FLUCSUB)
** Application transparent conversion (FB/VB, TXT, CSV, XML, ASCII/EBCDIC/UNICODE), compression (GZIP, BZIP2, XZ, ZIP), encryption (PGP), encoding (BASE64/32/16) and transfer (SSH)
** FILE DD DSN=&SYSUID.TEST.DAT,DISP=SHR,SUBSYS(FLUC,'read.text(file=ssh://user@server/test.gz) format.record(recf=FB recl=80)')
** Improve DCB handling to simplify usage of subsystems
*** Add function to determine DCB parameter for DD names
*** If DCB parameter is provided, plausibility check is performed when opening
**** If a DISP=OLD/MOD allocation is used with DCB parameter in FALLOC object, then the parameter must now fit with the existing file
***** In the past these parameter are ignored and only used if DISP=NEW required
* Add support for anti virus scanning in access method for original data
** Arbitrary anti virus solutions can be used by implementing your own service provider interface (FAVE) that can be used in the AV conversion component (CNVAVS)
** A default implementation using a ClamAV daemon is provided
** Usable on FLAM APIs, the subsystems and the utility (FLCL)
** The solution conforms to PCI DSS requirement 5: Malware and Anti-Virus
* Don't load environment variables as part of a function call anymore
** The application developer needs control over the environment
** Add new functions (fcbenv, FCRENV, flucenv, FCUENV) to APIs to load the environment before the first API call
* Add several solutions to simplify code page issues on EBCDIC systems
** Support escape sequences for critical punctuation characters for EBCDIC systems (&HSH;)
** Parts of a command string can be in a charset different from the local charset by prepending a CCSID in an escape sequence (e.g. &1140;)
** Support environment variable (CLP_STRING_CCSID) for command string interpretation
** Support for entering hexadecimal values (&xFF;)
* SSH support for FLAMFILEs (remote reading and writing of FLAMFILEs through FLCL, FLUCSUB, FLUC, byte-, stream- and record interfaces)
* Support flucFS on z/Linux
* Update of libssh to version 0.8.3
** New cipher supported: chacha20-poly1305@openssh.com
** Support for curve25519-sha256 key exchange method as alias for curve25519-sha256@libssh.org
** Support for new rsa-sha2 hostkey algorithm
** Added support for diffie-hellman-group18-sha512 and diffie-hellman-group16-sha512
** Improved performance and reduced memory footprint for sftp
* Reduce memory consumption (in general and specifically for XML format in table component)
* Improve error trace by temporary file name if a ZIP archive is reorganized (for better error determination)
* z/OS and z/USS code are now built for minimum target zOSv2r1 (still for minimum architecture 8 and tune level 11)
* On Unix-like and Windows systems, environment variables are read from the file ".stdenv" in the working directory or (if it doesn't exist) in the home directory
* Support default values for empty strings in integer and float components (including empty indication)
* Improved handling of temporary files (required for ZIP archives and others)
* Use command mode as default for FLCC commander
* Support RECFORMAT=ORG(inal) in record preparation
** Use the original record format when writing
* Enforce DISP=NEW allocation if the RENEW is switch enabled
** No failed DISL=OLD allocation in front anymore
* Support up to 254 sub system parameters with maximum length of 67 characters
* Support re-keying of secure key rings in FKM5-PGP software implementation
* Improved log messages for I/O components (especially for debug log)
* Support first valid letter (if alpha) of the system symbol '&SYSCLONE' as default 'ENVID' on z/OS
* Add new function fcbsym() and FCRSYM() to byte and record interface to load/set environment variables, JCL and system symbols
* Support two new environment variables for better Micro Focus EDZ support concerning FCRSYM() and FCRENV()
** FLAM4MF_STATIC_SYSVAR - static system variables are read from filename into the environment (Windows or Linux filename in local character set, Syntax: SYMBOL=value)
** FLAM4MF_DYNAMIC_SYSVAR - dynamic system variables are read from filename into the environment (in conjunction with the JCL User Exit of MF-EDZ to add dynamic variables)
* Support additional 'DD:SYSVAR' for Micro Focus EDZ environments (requires EBCDIC support on ASCII system and separation with semicolon)
* Support EBCDIC and ASCII auto detection and additional separation by semicolon in all files defining environment variables
* Add new GUI tool 'flgt' including editing formatted, compressed and encrypted files on Windows and Unix-like systems
** The GUI tools 'flinfo', 'flview' are now included in the new tool 'flgt'
** All functions can be used via parameters (flgt -c=view filename)
** The default command of 'flgt' is edit
* Support "write.dummy" as abbreviation for "write.binary(file=dummy)"
* Add new PGP FKM5 parameter "newpass" to re-encrypt secret key rings
* Add more plausibility checks about row definitions for XML formats
* Support charset and endianness 'LOCAL' in addition to SYSTEM
** LOCAL represents the real character set or endianness on this machine (Windows: CP1252/ASCII with little endianness)
** SYSTEM represents the logical default character set or endianness defined in the environment (MF-EDZ: IBM1141/EBCDIC with big endianness)
** The system endianness is no longer the local endianness (in MF-EDZ environments)
* Add feature check for SSH connections and hash calculations
* Add fallback and warnings if test of random number generator fails
** If seed has not enough entropy, a warning is written to the log (results in completion code 1)
** On z/OS, this can happen if ICSF is not active
* Bugfixes
** Correct stack allocation for big numbers (lple64 failed, because the 1280 byte are not enough)
** Don't use unsafe function tmpnam() anymore
** Fix several issues concerning dia-critical character support for EBCDIC
*** Fix inconsistent default and system code pages (result in strange character conversion if no LANG variable defined)
*** Correct all printouts to conform to the defined system CCSID on EBCDIC systems (see environment variable LANG)
*** Don't use dia-critical characters before the environment is loaded
*** Implement a real singleton for critical punctuation characters which is loaded only once
** Fix completion codes (some times SYSTEM (36) and MEMORY (48) errors are mismatched)
** Use correct delimiter if CX7 files encoded in ASCII or EBCDIC
** Correct character set in internal default value conversion
*** Prevent error in MF-EDZ support at conversion/mapping string to integers and floats
** Ensure removal of written file if error in close functions occur (required if virus is found)
** Correct handling of FLMRTC_AIR (prevent wrong data after re-alloc with pointer change at delayed read from memory)
** Fix missing end-of-table (EOT) for the first record during in-memory conversion if the first record does not match the first row definition
*** This wrong behavior only occurs at the first record if single record conversion with EOT handling done
*** The fix will always result in a FLMRTC_EOT at the beginning if the ENDOFT switch is enabled
** Increase final random check verification from 32 to 64 bit to prevent clashes
** Move hash verification from close to final function (to ensure removal of written file if hash verification fails)
** Ensure XML prolog is in the first line of each XML document
** Correct prolog and comment handling in TABXML (print comment and prolog in front of each table)
** Fix directory walk on statically allocated PDS member (FILE='DD:INP(*)')
** Fix replacement of environment variables in parameter strings (correct support for more than one <XXX>)
** Fix removal of PO datasets if no member is left (check if the file really is a PDS)
** Correct FLVEDIT line command on z/OS if an error occurs while writing (PF3 will always result in a save (no cancel possible anymore))
** Use original record length when reading FLAMFILEs if platform is FL5 (FLUC)
** Correct adjustment of block size in record I/O component
** Don't write DCBs in record I/O component to inverse command on systems with catalog (z/OS)
- 0000862: [2.2 Subprogram FLUC (CONV)] Support of more then one XML document per file (Tilo Eckert) - resolved. - 0000920: [3. Subsystem, Driver, Plugin, SPI] Add GUI tool like FLVEDIT of ISPF to edit encrpyted, compressed and converted files on windows (Mykhailo Moldavskyy) - resolved. - 0000951: [2.2 Subprogram FLUC (CONV)] Add password change for keyrings to PGP support (Falk Reichbott) - resolved. - 0000954: [1.2 FLCC] Support write.dummy (Falk Reichbott) - resolved. - 0000704: [3. Subsystem, Driver, Plugin, SPI] Add a FLUC I/O subsystem to write transparent different formats for and to different platforms on z/OS (Falk Reichbott) - resolved. - 0000950: [2.2 Subprogram FLUC (CONV)] Support anti virus scanning at read and write with FLAM (Falk Reichbott) - resolved. - 0000825: [2.2 Subprogram FLUC (CONV)] SSH-Support for FLAMFILEs (Tilo Eckert) - resolved. - 0000946: [4. Element, Byte, Record or other API] Don't load environment (SYSVAR, STDENV) in API functions (Falk Reichbott) - resolved.
* Add first pre- and post-processing filters to table support
** For type conversion
*** TABLE(... ROW(... COL(... PROCESS.BINARY/STRING/INTEGER/FLOAT(FROM() TO()) ...)))
** Masking of strings (e.g. for card holder data, conform to PCIDSS)
*** TABLE(... ROW(... COL(... PROCESS.MASK(CHAR=ASTERISK HEAD=6 TAIL=4 CHANGE=INNER) ...)))
** FCPE service provider interface (SPI) for custom validation, tokenization, manipulation, encryption, ...
*** TABLE(... ROW(... COL(... PROCESS.EXIT(LIBRARY='libfcpe' FUNCTION='colenc' PARAMETER='keylabel') ...)))
* Support for writing to multiple targets in parallel on byte, record and stream interfaces
* The URL schema file:// can now be used to specify local local files using URL syntax
** Special characters may need to be URL-encoded (%xx)
* A CCSID can be passed through filename parameters in URLs and normal filenames (file='filename.txt/&IBM1141')
* URLs can now also be used for parameter files including remote access via SSH and character set conversion with auto detection
* Add PGP public key v5 support
* Calculate PGP KeyID and fingerprint only once and keep this in public key structure
* SSH with ECC keys now works on all supported platforms (was disabled on z/OS / USS due to incorrect calculations)
* Add new class libraries for memory to memory conversion capabilities of our byte interface for Java
* Support thousands separation when writing integers and floats
* Support new FCRCPYL function in FLUC record interface to support MF-EDZ workaround also for the log stream
* Add re-structured COBOL sample for SEPA mass and instant payments with the FLUC record interface
** Includes workaround for MF-EDZ pointer issue (used in Linux and Windows packages, comment out in z/OS packages)
* If no delimiter is provided for format DLM in table support when writing, use charset dependent default delimiter
* Explicit call of flush function now results in a forced flush (creates an empty file if no data is available)
* Handle ASA and MCC correctly if print control retain is used when reading with or without record preparation on file to memory APIs
* Support record preparation when reading from memory (understand records with print control characters (ASA/MCC) and/or slot numbers (RRDS))
* Add Cobol copy books SOFLCREC.cpy and SOFLUCUP.cpy to Linux and Windows packages
* Add trace facility (FLAM4MF_TRACEFILE) for Microfocus EDZ stubs
* Implement pointer conversion in MF-EDZ stubs (required if AMODE defined and MF-EDZ is fixed)
* Add minimal length check per column for external representations of data fields in table support
* Support new length+pointer data formats as FORMAT=VAR for writing to or reading from memory in table support
** Pointers to data are written/read instead of the data itself
** Field order and padding can be controlled by keywords: ALENPTR/PLENPTR/APTRLEN/PPTRLEN
** Null indication flag to improve handling of columns containing default values
* Add new functions FCRCPY/FCRPTR for FLUC record interface for MF-EDZ workaround if the new table formatting VAR is used
* Improve Linux and Windows packaging for MZ-EDZ installations (see corresponding install.txt)
* Support Linux on IBM Power architecture in big and little endian format
* Support NOCHEK parameter in default objects of table support
* Remove auto pass through from flucFS if no matching read configuration found
** For pass through a last read specification for file pattern * with format.bin() read.bin(file='') must be add to flucFS configuration
* Add FKM5/E strings to FLCC commander
* Make FLAM command visible in FLCC
* Move inverse row specification from info to debug logging output
* Bugfixes
** Fix incorrect error handling in close, flush and delayed open function of byte and record interfaces
** Fix get table name function for memory converter (don't gives an error if delayed open not yet done)
** Correct password replacement with '***SECRET***' if CLEP is used
** The "defaults" object in table conversion had an incorrectly named selection value (INT instead of BIN) for "intfmt" and "fltfmt"
*** Adjustment of conversion strings may be necessary if you use one of these
** Fix some memory leaks in flucFSconfig tool
** Fix 0C4 in open for write if function FCROPN was used in MF-EDZ environment
** Prevent wrong error message in open for writing if the RENEW switch is activated, the file does not exist and something else goes wrong
** Fix determined record length for pseudo ASA/MCC datasets (FRA60 must be allocated with LRECL of 61 but RECL is 60)
*** For fixed-length record formats, this bug results in a length error when writing
*** For variable length record formats this bug results in wrong allocated record length
*** The bug was introduced with FLAM v5.0
*** The ASA and MCC support for non ASA/MCC dataset formats now works correctly
** Increase record length by 1 in FIO.REC() if records contain print control characters (retain) when writing and record format does not have control characters set
*** Prevent 'record too long' errors and the record length must no be set explicitly anymore
** Properly reload array contents in FLCC/flucFSConfig
*** If more than one object or overlay was defined (array definition) then all these definitions were lost
** Don't use charset detection in FMTTAB if none or a binary format was specified
** Improve error message in binary converter with base and charset if encoding or decoding failed
** Fix memory leak if GETPROP built-in function of FLCL was used without a command as parameter
** Fix segmentation fault if default SEPCHR was used in table support when reading
** Get field enclosing method NONE up and running at write of CSV data
** Write warning to log if FLAMFILE not read complete in info function
** Fix FLCC manual view if more than one level in bullet lists
** Fix memory exhausting issue in flucFS
** Fix parsing issue for XML in table support (<elm></elm> differs from <elm> </elm>)
** Fix writing issue for XML in table support (error if the first branch only contains optional fields)
** Empty or invalid strings for integer or floats results now in a format error
*** An empty string for integer was 0 and a empty string for floats was an interface error
*** 'nnn.' for floats results now also in a format and no longer in an interface error
*** '.nnn' for floats results still to '0.nnn'
- 0000923: [1.1 FLCL] Add support for NULL indicator to table support (Falk Reichbott) - resolved. - 0000944: [1.2 FLCC] Add FKM5/FKME CLP strings to FLCC command list (Ulrich Schwab) - resolved. - 0000900: [2.2 Subprogram FLUC (CONV)] Add cabebility for customers to write a own pre- and postprocessing exit for columns in table support (Falk Reichbott) - resolved. - 0000933: [3. Subsystem, Driver, Plugin, SPI] Make behavior configurable if no read specification found for a file in flucFS (Ulrich Schwab) - resolved. - 0000924: [4. Element, Byte, Record or other API] The packaging for MF-EDZ platforms must contain each open and singelton Funktion as DLL/SO of FCRRECMF (Ulrich Schwab) - resolved. - 0000927: [1.2 FLCC] Support a new data type for variable length fileds (strings/binaries) with length and pointer (Falk Reichbott) - resolved. - 0000922: [4. Element, Byte, Record or other API] Pointer to pointer return values are not usable in MF-EDC Cobol programs (Ulrich Schwab) - resolved. - 0000913: [1.1 FLCL] Add check for empty data elements in tale support (Falk Reichbott) - resolved. - 0000940: [1.1 FLCL] Write record with set RECDLM parameter should automatic set recordformat to delimiter (Falk Reichbott) - resolved. - 0000939: [1.2 FLCC] Support character set of a file in URL syntax (requires %-encoding and local file schema) (Falk Reichbott) - resolved. - 0000936: [4. Element, Byte, Record or other API] The print control handling does not work a write on byte and record interfaces (Falk Reichbott) - resolved. - 0000935: [4. Element, Byte, Record or other API] Add trace capability to MF-EDZ stubs of record interface (Ulrich Schwab) - resolved. - 0000929: [1.2 FLCC] Support group separator for integer and float string representations (Mykhailo Moldavskyy) - resolved. - 0000934: [4. Element, Byte, Record or other API] Rename FCRGELL in FCRLOCL and implement FCRGETL as copy function (Falk Reichbott) - resolved. - 0000912: [4. Element, Byte, Record or other API] Support multiple writes on the byte interface (Falk Reichbott) - resolved. - 0000928: [1.2 FLCC] The replacment of passwords with '***SECRET***' does not work anymore (Falk Reichbott) - resolved. - 0000905: [2.2 Subprogram FLUC (CONV)] Support ECC for SSH (Tilo Eckert) - resolved.
* Add trace facility (FLAM4MF_TRACEFILE) for Microfocus EDZ stubs
* Fix 0C4 in open for write if function FCROPN used in MF-EDZ environment
* Prevent wrong error message in open for write if the RENEW switch activated and the file does not exists and something else goes wrong
* Implement pointer conversion in MF-EDZ stups (required if AMODE defined and MF-EDZ is fixed)
5.1.17-20755
* Correct password replacement with '***SECRET***' if CLEP used
* Fix get table name function for memory converter (don't gives an error if delayed open not yet done)
* Support new FCRCPYL function in FLUC record interface to support MF-EDZ workaround also for the log stream
* Add new structured version of Cobol sample SOFCRIPS for SEPA mass and instant payments
** Includes workaround for MF-EDZ pointer issue (used in Linux and Windows packages, comment out in z/OS packages)
* Add Cobol copy books SOFLCREC.cpy and SOFLUCUP.cpy to Linux and Windows packages
* Fix incorrect error handling in close, flush and delayed open function of byte and record interfaces
* Support ECC keys for SSH on USS and z/OS (fix XLC issue in libssh code)
* Fix memory leaks in flucFSconfig tool
5.1.17-20714
* Add new system variable &FLREUCEE. to disable (=OFF) reuse of runtime environment on z/OS for ZEDC support in FLAM4 components
** This prevents user abend 4088 reason code X'63' if that NAB back chain is used depending on CEEPRM configuration (RPTSTG(ON))
* Add MF-EDZ stubs for logging facility control to record interface (FCROPNL/CLRL/GETL/CLSL)
* Fix few conversion issues in MF-EDZ stubs if EBCDIC used for parameter strings
** Add conversion for table names (FCRGTN and FCRSTN)
** Converted string parameter don't limited in length anymore
* Support conversion handle of previous conversion in open of next conversion like a file handle
* Support DD:Names and DataSetName (//'...') for all kinds of files on Windows and Unix in MF-EDZ environments
** Since previous version only the input and output files are supported
** There was no support for report, signature, hash, parameter or other kind of help files
* Byte/record interface no longer return EOT if the set-table-name-function was used to set the current table format
* Add COBOL example (SOFCRIPS) for SEPA mass and instant payments via record interface
* Add security checks when extracting files from archives (FLAMFILE, ZIP) or other containers (PGP, gzip)
** Prevents directory traversal attacks through specially crafted archives
* Improve error message in FIOFL4 if FLMOPN failed with -1 as handle and no reason is given
* Fix FLCC commander after QT 5.11 adjustments (command tree not usable)
* Add FMTTAB (table support) feature to demo license
5.1.17-20572
* Improved performance (10%) and reduced memory consumptions (40%) when converting tables to and from XML
* Fix handling of last block in in-memory conversion
** This bug could result in the last record from one block to be placed at the beginning of the next block
* Fix last block handling in XML parsing component for table support (in conjunction with previous bug)
* Correct support for concatenated XML documents
** Reading no longer fails if a block ends within the first token of the next XML document
* Improve format detection in table support for fixed and variable formats
** If character conversion failed, it is now handled as format error
* Extend error trace if XML parsing failed with context information
** Print 64 byte string starting at the error position
* Support DD name and data set name access on USS again
** Since SSH support was introduced the DDN ('DD:name') and DSN (//'data.set.name') support on USS failed
* Add table with usage recommendations for read/write objects (read.record() write.text()) of CONV command in FLCL book
* Correct formatting of byte and record interface documentation (some code blocks were not closed)
* Improve key access if OP signatures are used in PGP component (only use KID for faster access)
* Allow JCL/SYSTEM symbols with and without '&' at the beginning (<&xxx> == <xxx>)
** In the previous builds, JCL symbols with '&' failed and system symbols required '&' at the front
* Fix bug in FLUCUP which could result in suppressed error codes which might have resulted in truncated files without an error
* Fix crash of FLCC with Qt 5.11
5.1.17-20431
ATTENTION: Non backward-compatible changes in version 5.1.17
To improve performance, reduce memory consumption and support thousand
separators at read, the integer and float data types don't support the
whitespace parameter in external string representations anymore. This
could require adjustment of parameter strings for integer and float
data types if format is string. The new hard coded whitespace handling
remove all the whitespace and the thousand separators from the string
but don't add any zeros (',4' to '0,4') to build a valid number. If
this feature of 'whitespace=number' is required, you can use the
whitespace handling of the string data type for numbers and add a
pre-processing step (available with revision 5.1.18) to convert the
string type to an integer or float data type.
* New FLUC subsystem for Unix systems supporting FUSE
** A new executable to mount a physical directory to a logical
** Converts files transparently while reading/writing from/to the filesystem
** For each file name pattern, a FLUC write and/or read string can be specified
** These strings can contain any kind of conversion supported by FLUC
** The I/O subsystem can be used to convert, compress and encrypt certain files
** In the write definition you can define, if a transparent/inverse read over the logical few possible or not
** The file in the physical folder (e.g. PGP files) can then be handled like any other file (transfer to partner, backup, ...)
** The solution was built to enhance standard applications (e.g. SAP) with transparent file conversion, compression and encryption features
** The flucFS (file system) can be used as transparent infrastructure to control access and format of certain critical files
* Add XML format to table support based on a hierarchical path syntax
** Read tables from structured or unstructured XML formats (e.g. convert SEPA transaction to a fixed-length record)
** Write tables as structured or unstructured XML format (e.g. convert a fixed record to a SEPA transaction)
* Support records up to 1 MiB in length (required for SEPA-XML to fixed-length record conversion)
** There is no real limit anymore, but for plausibility checks the 1 MiB is used
** This size can be expanded to a maximum of 1 GiB
* Support additional columns with default values at write (required for XML support)
** Can be used to insert columns to a read table at write
* Support of concatenated XML documents with FMT.XML()
* Add NFD/NFC support to character conversion
** Normalization of UNICODE for compare and/or sort
** Combined character support with auto detection if conversion to single bytes character sets (e.g. Latin1, CP1252 or IBM1141)
* Support setting file permission bits (using chmod) in I/O components (EXTATR)
* Support system symbol replacement on z/OS ('<&LDAY>') in CLEP strings
* Extend default record length by delimiter length if record format TXT or DLM is used
* Add GM/LCYEAR2 string literal for year without century (YY) to build file names and other kinds of strings
* Add new keywords 'SYSTEM' or 'LOCAL' for CCSID to ensure the local system-specific character set
* Improve software random number generation (not relevant for HSM usage)
* Reduce memory consumption for character conversion
* Reduce CPU and memory consumption when using base encodings
* Read environment also from '&SYSUID..STDENV' on z/OS
* Add completion code message to FLCL with version and builds number
* Version and build number are separated with a hyphen instead of a dot
* If pattern substitution in filenames ([xyz]) results in a zero-length filename, an error is returned instead of writing to stdout
* Make buffer for usage report dynamic in size to support long job name lists
** The buffer was limited to 64 KiB which could result in incomplete reports
* Support for digit grouping by a delimiter (thousands separator) when reading integer and float data types
* Support keyword DUMMY for files at read like a read of an empty file
* Make timeout for SSH operations variable (default still 30 seconds, but now you can define lover (1) or hihger values (68 years))
* Add logging facility control function to byte, stream and record interfaces
** Required for table support to catch warnings (required fields or branches, missing column specifications, ...)
** Open and close function for memory logging, getter for log buffer and maximal codes and reset functions to clear anything
* Provide memory to memory interface to use table support for transactions (online processing)
** Conversion between fix, variable structures and CSV and XML formats
** Useful for example for instant payment SEPA transactions
* Extent memory to memory interface to support compression, encryption and encoding schemes
** Provides complete read and write capabilities of FLUC (read a GZIP data stream in XML (UTF-8) and write CSV in EBCDic as PGP stream)
** Useful to produce for example GZIP oder PGP attachments for mails inside of an application
** Requires delayed open at read from memory to support all auto detection capabilities of FLUC
** Support new entries for intermediate memory to memory conversion (fcbrunc/finv or FCRRUNV/FINV)
** Support new entries which return a pointer to the converted data (fcbconp/runp/finp or FCRCONP/RUNP/FINP)
* Support record preparation (fixed length, ASA and MCC control character) at format.record() for API
** Required for FLUC-Subsystem on z/OS to provide fixed length data sets and record formats with ASA or machine control character (MCC)
* Bugfixes
** Fix wrong headline handling if record orientation used in table support
** Fix potential segmentation faults in table support
** Fix wrong realloc for element converter (*piOutLen was not set)
** Fix encryption of final PGP block if literal packet is less than a symmetric block length
*** Empty files with short filenames not longer result in a NEED_DATA error
** Fix error handling when closing ZIP files (don't lose error if re-organization fails)
** Add missing final call in FLUC-UP to ensure completeness of written data (required for XML in table support)
** Fix segmentation fault in whitespace handling for floats and integer data types
** Correct wrong error handling in file I/O components (real error at close was lost)
** Fix [cut-n] filename pattern rule when using negative numbers
** Fix segmentation fault if record length for FLAMFILEs incorrect (RECSIZE<80)
** Fix wrong mapping of record length for FLAMFILEs 8RECSIZE9 and input files (INRECSIZE) in FLAM command of FLCL
** Add final handling of input data if end of file determined after the first block (as part of the open function)
** Fix wrong handling of block size in PGP info function (some times the PGP dump was not provided for a PGP file)
** Return required length if FLMRTC_LEN error return after call to FCRGET
** Correct format detection at read (isFinal then FLMRTC_FMT instead of FLMRTC_LEN)
- 0000915: [1.1 FLCL] Read write from USS to zOS dataset not possible (Falk Reichbott) - resolved. - 0000892: [4. Element, Byte, Record or other API] Add support for ASA and machine control character to format.record() on byte interface (Falk Reichbott) - resolved. - 0000888: [3. Subsystem, Driver, Plugin, SPI] Support an application transparent file conversion, compression and encryption solution like z/OS-Subsys for open systems (Ulrich Schwab) - resolved. - 0000896: [4. Element, Byte, Record or other API] FMT.TAB as Memory to Memory Interface (Mykhailo Moldavskyy) - resolved. - 0000907: [2.2 Subprogram FLUC (CONV)] Add XML format to table support (Falk Reichbott) - resolved. - 0000874: [2.2 Subprogram FLUC (CONV)] Support of combined character in UNICODE (Mykhailo Moldavskyy) - resolved. - 0000932: [3. Subsystem, Driver, Plugin, SPI] The match patterns does not work in each case within flucFS (Ulrich Schwab) - closed.
* Improve usage log summary
** Fix wrong CX7/CX8/VR8 sum
** Don't count FL4 stuff in ZIIP/ZEDC/CPACF sums of other components
** Add sum for remaining ADC load if ZEDC used
** Add summary for FIO, CNV and FMT read and write CPU times
* Correct dynamic allocation and de-allocation on z/OS
** Fix error handling at DISP=NEW (write of file which does not exists) if allocation failed
** Check return code if dynfree failed at close and ensure correct DD name for deallocation
*** The deallocation failed in the reopen case which results in an allocation table overflow
*** Prevent segmentation fault if more than 1000 dynamic allocations done at reopen (directory walk)
** Add check to each write function if the internal write handle is NULL
*** Prevent segmentation fault after reopen failed
** Use only DD name for dynfree to ensure deallocation in each case
5.1.16-18986
ATTENTION: Threshold procedure for ZEDC support in FLAM results in damaged FLAMFILEs in previous builds
Please use this build to prevent damaged FLAMFILEs if expansion of ZEDC larger than the defined threshold
* Fix incorrect pointers in element list of table support result in segmentation fault or data damaged in rare cases
* Improve character set detection
** UTF-8 with mainly Cyrillic letters are not detected as EBCDIC anymore
** Better separation of binary data from UTF-8 encoding
** More significant EBCDIC detection using a higher weight for letters and numbers
** No character set detection in FMTTXT if character set known from CNVCHR
* Adapt window size to length of output in GUI tools
* Support FLCL configuration for FLAM4 components in FL5 project (ZEDC support)
** Environment variable defined over FLCL are now recognized by FLAM4
** Better performance if ZEDC used for FLAM4 ADC with FLCL
* Fix wrong error handling in ZEDC support for FLAM4
** If threshold defined for ZEDC support and the expansion of the the ZEDC compression exceeds this value a damaged FLAMFILE is build
** zlib returns OK (0) and set avail_in to 0 if Z_FINISH defined and writes the complete output buffer and stores the rest of data in the handle
** this results in ZEDC usage in case the threshold is effected (output buffer too small) and not in the use of the original ADC algorithm
** with the result, that the first segment and all followed segments are damaged (to read such a file a special tool could be provided)
* Activate by default SKPBIN in READ.TEXT() to prevent character conversion if binary data detected
** Add a new switch "CNVBIN" to enforce character conversion if binary data detected
5.1.16-18904
* Add COBOL Samples to Windows and Linux packages for MF-EDZ customers
* Improve time value handling in FLCC commander (new dialog provided)
* Few additional performance improvements for table support (30% less CPU)
* Correct error handling of FLMREC in MF-EDZ environment
* Add new sample COBOL source for a FLAM-Subsystem for MF-EDZ environments
* Support "Issuer Fingerprint" in PGP key, added in GnuPG 2.2
* Extent usage log on z/OS to evaluate job names and use separate components for FLAM4
** The new switch "JOBNAME" can be used to replace function (READ/WRITE) by job name on z/OS
** MATSEG for each FL4 load are now replaced by: FL4CX7/CX8/VR8/ADC/EDC/NDC
* Extent FLCC commander to support integer, float and string expressions
* Fix segmentation fault in Windows pop up GUI tools
* z/OS and z/USS Code are now optimized (TUNE(11)) for z13 CPs (minimum is still z10 (ARCH(8))
* Adjust OBTAIN call in FLAM4 components to fix EAV-TRACKER issue in z/OS
5.1.16-18817
* Fix performance issues in character conversion module
** The better check of valid code points was not implemented well
* Show license error in dialog box of FLCC
* Add sample job to produce usage report for the last quarter (JOBLIB(FLUSAGE))
* Correct syntax of object array in overlay in FLCC commander
5.1.16-18756
* First version of table support (FB<->CSV<->VB)
** Reads files containing one or more tables and splits records/rows into several columns
** Columns can be read with great flexibility using various methods (fixed length, length field, binary delimiter or CSV)
** Data types can be assigned to columns to enable column conversion (binary, string, integer, float)
** Each data type supports various representations (character set, endianness, BCD, ...)
** More features: auto-formatting on write, reordering/removal of columns, pre/post-processing filters, multiple tables per file
** This version of table support is prepared for tables in hierarchical formats (TLV (e.g. ASN-1), TVD (e.g. SWIFT), XML (e.g. SEPA))
*** Support for Tag-Value-Delimiter (e.g. Swift) and XML files is planned for upcoming revisions
** Tables can also by read through our APIs which enables reading rows into custom data structures in custom applications
* Character conversion now supports n to m mapping of code points
** This allow to define input assistance (tagging) for Unicode in single byte code pages (EBCDIC)
** It also enables NFD/NFC and complex best fit mappings for the String.Latin subset (see NPASNFD/NFC)
* New REMOVE switch in each write specification allows to delete the output file if an error occurs
* FL_PLATFORM is now also used for system delimiter, EBCDIC/ASCII separation and byte order definition (required for MF-EDZ support)
** If environment variable 'FLAM4MF' starts with 'IBM' then a EBCDIC system (FL_PLATFORM="ZOS.B") is assumed in the FL5 components
** Support for big and little endian for integers in element structure of byte and record interface
** Delimiter and other special characters are now defined based on environment settings if CHRSET_SYSTEM is used (FMT.TXT, CNV.BAS)
* Record Interface: Parameter 'del' was renamed to 'success' at FCRCLS to be uniform with FLCBYT
* Use ACEE control block of current TCB to determine the login user id on z/OS (instead of POSIX functions getuid()/getpwuid())
* Ensure PKCS#11 support (OpenPGP) works with Thales (former nCipher) nShield network HSM
* Add DSA support (if possible (key ring and PKCS#11 not for ICSF/CCA)) to OpenPGP components
* Remove FL5 archive parameter from FMT.RED object of XCNV (simplification)
* Change printout of FLCL UTIL RUN.LIST command to accessible URLs or local files
** Since the introduction of SSH support (since 5.1.13) the LIST function added "local file" or the remote location in parenthesis behind the filename
** This change has been reverted to the behavior of version 5.1.12 and below due to compatibility issues
** For remote files, a complete URL in FLCL syntax is now printed
* The FLUSH flag now ensures that written data is persisted to the storage device before the file is closed (fsync)
** For remote connections the availability of this feature depends on the protocol and remote server
*** SSH: Server must support the fsync@openssh.com extension (OpenSSH >=6.5); INFO message is printed if unsupported
* libssh updated to latest version (0.7.5)
* FKM5-CCAPGP: Key template replacement is now case insensitive and implements better key template checks
* New 'remain' flag for z/OS datasets with dynamic allocation (prevents overwrite)
* Support LRECL independent record length for VS/VBS datasets on z/OS
** Large record support (>32760) is not possible based on LE limitations
* Add log entry for FLAM4 ZEDC support (use of IBM ZLIB for ADC compression)
* Support leading spaces in file and format strings for byte, stream and record interfaces
** The first character of the file string must be 'R', 'I', 'W' or 'O' to know if it is for read or write)
** Space in front of the first character is now supported, trailing whitespace is ignored by CLP
** This is required to support simple handling of parameter files for the FLUC subsystem
* A state string length of 0 at FCROPN indicates that no state is requested when reading (no longer returns a length error)
* New function FCRCLR in record interface to clear background error information
* The calculated inverse command now only contains the first FIO specification to prevent syntax error when triying to read using this string
* A threshold for the maximum expansion of compressed data can be set if ZEDC is used for FLAM4 archives
** Set the environment variable FL_ZEDC_SUPPORT to the maximum expansion in percent
** Recommendation is a value of 60 (FL_ZEDC_SUPPORT=60)
* Support the FLUC record interface (FLCREC) in Microfocus EDZ environment
* Improve overall performance by better optimized code
* Bugfixes
** Fix missing initialization of statistic structures in directory walk when writing
** TO=YOUNGEST for usage report now equals 0 and not -1 (prevent error from CLP for unsigned values)
** Support 0x0D15 and 0x0D25 on EBCDIC systems as delimiter at FIO.TXT()
** Correct form feed character in EBCDIC from 0x22 to 0x0C (wrong interpretation in FIO/FMT.TXT() and charset detector)
** Change ExtArt in ExtAtr at XML output of info function in FIO.ZIP()
** Start record number with 1 instead of 0 to fix I/O error at load of RRDS without slot numbers
** Don't ignore the whole character if recursive transliteration stops, only ignore the non convertible remaining rest at character conversion
** Fix error and end of file detection in fcbgets()
** Fix wrong statistics output of FIO.FL4() if only one file in charge
** Fix wrong error handling in CNV.REC() which resulted in an interface error
** Fix several potential bugs detected by better static code analysis
** Fix usage of format argument of fcbinfo function of FLCBYT (only format list was available)
- 0000902: [6. Access Control, Logging and Errorhandling] Support JOB-NAME in usage log report (Falk Reichbott) - resolved. - 0000895: [b. Packaging] Add Cobol-Samples to Windows and Linux-Packages for MF-EDC development (Ulrich Schwab) - resolved. - 0000898: [1.2 FLCC] Better support for literal expression and arrays of overlays in FLCC commander (Falk Reichbott) - resolved. - 0000897: [1.2 FLCC] Show error message if access to license module failed (Ulrich Schwab) - resolved. - 0000791: [4. Element, Byte, Record or other API] Add support for FLUC record and subprogram interface for Microfocus EDZ (Ulrich Schwab) - resolved. - 0000866: [2.2 Subprogram FLUC (CONV)] Add table support to convert based on colums between CSV files and FB datasets (Falk Reichbott) - resolved. - 0000782: [2.2 Subprogram FLUC (CONV)] Support of conversion of tables, CSV files and FB datasets (Falk Reichbott) - resolved. - 0000875: [1.1 FLCL] support of tagged characters (Mykhailo Moldavskyy) - resolved. - 0000887: [2.2 Subprogram FLUC (CONV)] Dataformat detection and automatic compression algorithm selection (Mykhailo Moldavskyy) - resolved. - 0000881: [1.1 FLCL] Function for ZEDC usage on FLAM record interface (Rolf-Dieter Euler) - resolved. - 0000878: [2.2 Subprogram FLUC (CONV)] Support remove flag at write if write failes (Falk Reichbott) - resolved.
* Bugfixes
** Fix segmentation fault if XCNV called at output without FIO specification
** Fix wrong return code if connect failed in UTIL command
** Ensure and correct use of UTF-8 in XML-Dump
** Fix missing external attributes (file permission bits) for USS files on z/OS in ZIP archives
** Change ExtArt in ExtAtr at XML output of info function in FIO.ZIP()
** Correct know host file name in log output for SSH connections
5.1.15-17363
* Set DSN type automatic to LARGEFILE if primary tracks + (16 * secondary tracks) > 65536 on z/OS
** previous calculation was without the multiplication of secondary tracks with 16
** each file which can be a large file will be a large file by default all other are basic files except LARGEFILE is requested
* Bugfixes
** Fix missing initialization of of PKCS#11 mechanism
** Accept C_RandomCeed not support for PKCS#11 implementations
** Correct search of objects in PKCS#11 implementation
5.1.15-17354
* Bugfixes
** Correct handling of locale in float element converter
** correct SBUILD for assembler modules
** Fix bugs CCAPGP with trust store (PGP-keyring)
5.1.15-17337
* Support for packed BCD real numbers in element converter
* Reduction of memory consumption
* Most strings (e.g. paths) no longer have a length limit
* zlib updated to version 1.2.11
* Build time and update recommendation are now logged
* An info is printed to the log when the installed FLAM version is no longer supported
* A warning is printed if the license file is for a newer FLAM version than currently installed and the FLAM version is out of support
* Bugfixes
** Fix errors in float conversion of elements
** Correct handling of file prefix on z/OS
** Fix error in PGP file handling
*** Don't interpret last interim package of length 0 as first padding character
*** In very rare cases, this resulted in a file where the last part is missing when reading a PGP file
** When a public key file was passed to the SSH net object, building the filename of the private key was done incorrectly
** Fix MF-EDZ filename mapping in FLUC utility (FLCL CONV/XCNV/DIFF/...)
** Fix wrong mapping of state string in byte and record interface (RecLen, BlkSiz and KeyLen damaged)
** Correct test of ZEDC availability in FLAM assembler modules
** The system locale was not honored correctly when converting from/to floating point numbers, potentially resulting in an incorrect decimal separator
** Fix for potential segmentation fault in PGP component if body length of user ID exceeds 1024 byte
5.1.15-16696
* Return condition code 1 if there is a relevant warning in the log
** For backward compatibility, the mechanism can be deactivated with environment variable FL_RETURN_IF_LOG_WARNING=OFF
** On z/OS you can also use the system variable &FLRLOGW to change the default behavior
* Support exported JCL symbols like environment variables in FLCL control statements
* Support CX8/VR8 encryption in FLAM4 to provide a powerful VSAM encryption
** Use the same passphrase and FKME support known from MODE=ADC
** Includes FLAM-Subsystem support for application-transparent encryption
** Usable via FLAM record interface and FLAM/FLCL utility
* New element converter for INTEGER and FLOAT data types
** Can be used to convert data between arbitrary number representations
** Supports standalone conversion of arbitrary data through the byte interface (fcbconv() / FCRCONV())
** Can be used as converter when reading/writing elements (e.g. number conversion for XML documents)
** Supported number formats: Binary, BCD, String (binary/octal/decimal/hexadecimal)
** Conversions can be described by a new conversion string that specifies the FROM-TO-relationship:
*** "conv.integer(from(format.str(chrset=UTF8 base=dec whitespace=number)) to(format.bin(width=W32 endian=big)))"
** See examples (SCRLIB) SOFCREXV and SOFCRINT/FLT
* A user table for character conversion can now be directly passed in the command string
** To separate the string from the filename, the string must start with a colon
** USRTAB=':(00C4=0041,0045)(00D6=004F,0045)(00DC=0055,0045)'
** Useful for short tables or on API usage
* Prevent printouts if FLUCUP is used (behavior of subprogram interface has been changed)
** The string ":STDERR" must be provided as output file to get the log output written to STDERR
** ":STDERR" and ":STDOUT" are supported for all output files (trace)
* Produce a syntax error in inverse command if original call for inverse command failed
* Extent usage logging to separate FLAM4 ADC (ZEDC) from CX8/VR8 (ZIIP) load
* The known_hosts file can now be passed as parameter in the NET.SSH object (KNOWNHOSTS)
* The new verbosity parameter in NET.SSH object can be set to get more verbose debug log entries
* The environment variable FL_SSH_HOME_DIR can be used set a different SSH home directory (default: .ssh folder in user's home directory)
* Add journal support (linux) for and improve usage reporting
* Improve performance of PGP file detection
* Add index number to filename mapping ([indN] with N from 1 to 8 to define the length of the file index)
* Add random number to filename mapping ([rndN] with N from 1 to 8 to define the length)
* The keywords for the filename mapping are not case sensitive anymore ([name]==[NAME]
* Support constant expressions for value assignment (blksiz=64KiB+8 from=now-30day) in command strings
** You can use '+', '-', '*' and '/' and '(' ')' for calculations of numbers and floats
** Supports a lot of keywords for constant values (KiB, MiB, NOW, DAY, LCSTAMP, SnRAN10, ...)
** If a keyword is used directly behind a number, a multiplication is done (4KiB==4*KiB==4*1024)
** For strings, the addition and the default multiplication with an keyword are implemented as concatenation
*** file='file'S4RND10'.txt' or file=LCDATE'.'LCTIME+S2RND10'.txt'
* Support parameter files for arrays (KEYWORD '[=' STRING ']') on the command line
** encr.pgp(user[='userlist.txt'])
** Preparation to support file lists in the next revision
* Support environment variable replacement ('<'ENVAR'>') on the command line
** Can now be used in any place where a lexeme begins, to produce a valid command string
** Replaced by the CLP scanner before parsing
*** file=<HOME>/file.txt --> HOME replacement by CLEP
*** file='<HOME>/file.txt' --> HOME replaced by FLUC
* Support PGP key lists in XML format
* Use PS-VB-512 as default for non-existing simple files on z/OS
** File without dynamic allocation (report, key store, ...)
** Where the language environment (LE) is simply used to allocate the dataset
** The LE default was PS-FB-80 and is now changed to PS-VB-512
** Like the dynamic allocated datasets
* Support padding with binary zero in PGP keyring files (FB datasets can now be used for keyrings)
* Write only an informational message if file size cannot be determined with FSTATZOS instead of a complete error message
* Support dynamic strings and arrays in CLEP
* Memory usage significantly reduced
* The following limits no longer exist (i.e. are now unlimited)
** Parallel writes/outputs (old limit: 4)
** Parallel FIO specification of XCNV (old limit: 4)
** Environment variable definitions in XCNV (old limit: 32)
** Decryption specifications (old limit: 32)
** Key management specifications (old limit: 32)
** Conversions (old limit: 32)
* Support multiple time windows for usage reporting (allows peak hour analysis of FLAM load)
* If wildcards and the remove switch used a PO dataset is now removed if no member left in the library
* Support suppression of data elements which contain only whitespace when reading XML data
* Support suppression of processing instructions when reading XML data
* Add optional length error if data element length exceeds the provided length when reading XML data
* A maximum condition code (range) can be specified [MAXCC=[max][-min]]
** If the condition code, that would be returned without MAXCC, would be greater than 'max', it is reduced to the value of 'max'
** If the condition code would be smaller than 'min', the actually returned condition code is 0
* Support option QUIET and SILENT in command syntax to suppress printouts of command line executer
** EXEC PGM=FLCL,PARM='HUGO CONV=DD:FLPAR MAXCC=8-4 QUIET'
*** This example uses owner HUGO to call CONV command where each condition code
*** greater than 8 is reduced to 8 and value smaller 4 are set to 0.
*** Additionally, normal log output of the command line executer is suppressed (except error messages).
* Bugfixes
** Fix memory overflow for files with more than 32 ARMOR header lines
** Files were opened twice in info functions
** Selection lists in FLCC now always visible
** Improve detection of ARMOR headers
** Don't write to STDERR if system logging used
** Fix wrong inverse command if read from archive used
** Fix error for log stream with two qualifiers (FLAM.USAGE)
** Fix devision by zero (0CF) in usage report summary calculation
** Fix amount of processed records in FLCL INFO GET.USAGELOG()
** Fix handling of 0x0DC285 as delimiter
*** In rare cases, a 0x0D0A could be converted to 0x0DC285 in UTF8
*** Only 0x0DC2 was recognize as delimiter
*** The 0x85 was then the beginning of the next record
*** This resulted in a conversion error, because 0x85 is an invalid UTF-8 character
** Fix error handling in directory walk for files in the root directory
** Fix reason for different segmentation faults in FLAM4 for AIX 64 bit
** Improve file size determination for MVS datasets
*** There was an error for files with a large block size
** Fix FLAM 64 bit errors on sun Solaris
** Fix wrong XML data after read finalization (in rare cases)
** Fix potential segmentation fault if no delimiter is defined at record I/O
*** Set correct default text delimiter if record format is DLM
** Check license in FLAM subsystem correctly
** Correct error handling if a close operation after writing failed during directory walk
** Don't delete FLCL config file if option clear is used
** Use CCSID in read.record() to define correct character set for FIO.REC (to define the correct default delimiter)
** Correct whitespace handling for numbers (0000000 -> 0 and not an empty string)
** Fix amount of original bytes for FIO.FL4 statistics on z/OS (was always 0)
** Fix rare infinite loop in filename processing (possible with input output name mapping)
** Fix EOF when reading FLAMFILEs after an empty member is encountered
- 0000415: [y. General] Use dynamic allocated memory for URL/file names and arrays (amount of conversions) (Falk Reichbott) - resolved. - 0000865: [1.1 FLCL] Support replacement of JCL variables like environment variables in FLCL control statements (Falk Reichbott) - resolved. - 0000861: [2.2 Subprogram FLUC (CONV)] Support environment variable for ssh home directory (Falk Reichbott) - resolved. - 0000858: [2.2 Subprogram FLUC (CONV)] Delete PO datasets with flcl util run.remove() (Falk Reichbott) - resolved. - 0000855: [2.2 Subprogram FLUC (CONV)] If all members of a PDS(E) are removed then remove the PDS(E) also in directory walk (Falk Reichbott) - resolved. - 0000859: [1.1 FLCL] Add cabebility to analyse peak hours in usage reporting (Mykhailo Moldavskyy) - resolved. - 0000838: [2.2 Subprogram FLUC (CONV)] Support user table as string for byte interface (Falk Reichbott) - resolved. - 0000856: [2.2 Subprogram FLUC (CONV)] Character conversion failed from UTF-8 to Latin1 failed if 0DC285 is used as delimiter (Falk Reichbott) - resolved. - 0000854: [2.2 Subprogram FLUC (CONV)] Add index and randomnes to file name mapping (Tilo Eckert) - resolved. - 0000850: [2.2 Subprogram FLUC (CONV)] Support known_host file in NET.SSH support (Tilo Eckert) - resolved. - 0000845: [1.0 FLAM4 or older] Add flags to separate ADC from CX8/VR8 usage (Rolf-Dieter Euler) - resolved.
* Support for writing to up to four targets at the same time
** Targets can be local files, remote files or both
** The XCNV command supports up to 4 FIO definitions
** The CONV and XCNV command support up to 4 write/output specifications
*** The input file is read once, but multiple sets of conversions are applied, each written to a different target
* Support for detached PGP signatures
* INFO function now works for remote XZ, GZIP and BZIP2 files
* Add REMOVE/INDELETE/DELETE parameter to FLAM4 command of FLCL
* Add usage reporting to SYSLOG for each platform (if environment variable FL_USAGE_SYSLOG_IDENT is defined)
* Support usage reporting via FLCL INFO command (flcl info get.usagelog())
* Implement whitespace handling as part of the character conversion module based on code points
** STR.CNV() does not require local charset anymore
* IP address of the remote SSH server is now logged if the CONDBG logging flag is set
* Support for IPv6 addresses in file URLs (ssh://user:passwd@[112:3344:55:ABC::1]:4711//home/user/mytext.txt)
* Support for dynamic allocation of SYSOUT (including class and program/writer) on z/OS
* FLMACS reports the version of the license module and software modules
* Usage summary report now includes accounting information
* A CCSID can be specified for hash generation/verification if the hash is written to/read from a file
* Bugfixes
** Fix behavior of REMOVE switch for FLAMFILEs
*** Don't delete the FLAMFILE if not all members have been written successfully
** Correct behavior of REMOVE switch in directory walk and auto detection
** Correct FLCC command generation for overlay in overlay
** Reading small ZIP files via SSH failed due to positioning bug in SSH support
** Add missing close functions if INFO function failed
** Prevent huge memory allocations based on incorrectly calculated INICNT at record formatting
** Correct data volume in usage report from MB (1.000.000) to GiB (1.073.741.824)
** Fix processed bytes count in usage logging for FLAM4 matrix segment serialization
** Fix byte counts (>=2GB on UNIX/WIN >=4GiB on z/OS) for FLAMFILEs in statistics
** Default binary delimiter length changed to 0
*** Prevents wrong auto record formatting if UTF-16/32 is used
*** Prevents memory allocation error and segmentation fault
*** This wrong binary delimiter was stored in private GZIP/ZIP headers
*** Automated reading of these GZIP/ZIP files can fail if UTF-16/32 is used
** Fix segmentation fault after realloc with pointer change in FMT.REC()
*** Occurred in rare cases when records were produced based on length fields or binary delimiters
** Fix handling of PGP files if the last PGP packet is empty
*** In rare cases (empty last packet) the read of a PGP file could fail with PGP_FATAL_ERROR
** Maximum logical record length changed from 32751 to a record format dependent value on z/OS
** Correct dynamic allocation of large files on z/OS (DSNTYPE set to 'LARGE' if required)
** Fix reading of files with text and binary delimiters if read.record(recformat=dlm) is used
*** If the DECODE switch is not used and a record delimiter is given, then the default record format is now DLM.
*** For decoding of record-oriented datasets the decode parameter must be specified
*** Auto decoding is no longer performed if CCSID or write.char/text/xml() are specified
** A rest behind the last delimiter will not result in an error anymore in FMT/CNV.REC()
*** It will result in an additional record by default
*** Ensures the same behavior as FIO.REC(RECF=DLM)
*** The new default behavior can be changed using the XCNV command (ERROR, RECORD, IGNORE)
** Fix a memory deallocation issue in rare error situations when writing FLAM4FILEs
** Significant decrease of false positive format detection rate (PGP, Base, GZIP, BZIP, XZ)
** Correct handling of ARMOR header when trying multiple possible base decodings
** Support verification of binary and correct verification of hex values in CNV.HSH()
** Correct file size determination for large files on z/OS
** Fix wrong XML data after read finalization (in rare cases)
* Documentation
** New manual page for the REMOVE switch (important note on wildcards)
** HELP command output now also displays the description for the given command string and not only its children
** Improve appendix "properties" in FLCL manuals
** Improve docu and help output in FLCC (use converted HTML instead of plain ASCIIDOC)
** Add new chapter for handling of empty records to FLCL manual
** New look and feel for PDF documents
- 0000841: [2.2 Subprogram FLUC (CONV)] Sometimes wrong XML with a '1' somewhere in die XML stream is generated (Tilo Eckert) - resolved. - 0000849: [1.1 FLCL] Syntax error in inverse command since FLAM 5.1.12 (Falk Reichbott) - resolved. - 0000832: [2.2 Subprogram FLUC (CONV)] Support more than one write (CONV) or output (XCNV) specification (Falk Reichbott) - resolved. - 0000846: [6. Access Control, Logging and Errorhandling] Division by zero at determination of usage report (Falk Reichbott) - resolved. - 0000847: [6. Access Control, Logging and Errorhandling] FLAM.USAGE don't work as log stream (Rolf-Dieter Euler) - resolved. - 0000848: [1.1 FLCL] The NET.SSH object is not recognice with 5.1.14 (Falk Reichbott) - resolved. - 0000834: [2.2 Subprogram FLUC (CONV)] Support dynamic allocation of SYSOUT (Falk Reichbott) - resolved. - 0000830: [2.2 Subprogram FLUC (CONV)] Add IP-Addr to log for SSH connections (Tilo Eckert) - resolved. - 0000831: [2.2 Subprogram FLUC (CONV)] Support of more then one write target after conversion (Falk Reichbott) - resolved. - 0000744: [2.2 Subprogram FLUC (CONV)] Support for detached signatur generation and verification (Mykhailo Moldavskyy) - resolved.
* The z/OS and USS packages are now built for minimum version 1 release 13
* Remote file access via SSH (SFTP) including wildcard/directory support
** Supports public key and password authentication
** Communication parameters are passed via properties (net.ssh() object) or via URL (see below)
** URL: file='ssh://user:password@hostname:port//path/file.ext'
*** Only hostname and filepath are mandatory
**** e.g. file='ssh://hostname//path/file.ext'
*** Host and remote file path are separated by a slash
*** Absolute paths require another slash (i.e. // after the hostname)
*** The username defaults to the local username if not specified
*** Password authentication is attempted if a password is specified
*** Public key authentication uses the key pair in ~/.ssh/ by default
*** Replacements can be used in the URL
**** e.g. file='ssh://<cuser>@hostname//myfile.txt (<cuser> is replaced with lower case local user id)
** Record-oriented datasets are written remotely like the record I/O works on Windows and Unix systems
** Currently FLAM4FILEs are not supported (special user I/O required, will be supported later)
** Static allocations (DD:NAME) are not supported with remote access
** Most other kinds of files (log, info, report, ...) are supported via SSH
** Configuration, property, trace and inverse command files must be local
** See the documentation for the net.ssh object for more details
* New NoMkDir parameter to prevent automatic directory creation on write
* Add PGP key ring support (software implementation of RFC4880)
** Supports all symmetric and asymmetric algorithms available with OpenSSL (libcrypto)
** Key ring files are compatible to GnuPG and can be used by both solutions
** Extended PKCS#11 implementation which works like the software key ring solution
** Extended new CCA/ICSF implementation which uses a PGP trust store together with the PKDS
*** With this solution you can first migrate from GnuPG to FLAM-PGP (using the password protected key rings)
*** Then you can import the private key store to the CCA/ICSF device for more security
** PGP primary and subkeys may contain the same key (can be imported with a warning, but cannot be generated)
* Use FLAM4 system header to store file size, CCSID, binary delimiter and other parameters in the member header of a FLAM4FILE
** If FLUC is used to write and read a FLAM4FILE, then the CCSID, binary delimiter and file size for a member is known
* If the input file was set to STREAM, the output file now also defaults to STREAM
* Fix handling of multi disk values for ZIP64 files
* Reduce the amount of ZIP64 extra fields in the central directory file header to a minimum
* Correct large block support in FLAM4 for z/OS (required for tapes)
** Correction of block size smaller than 32k was damaged (works now like in the past)
** Correction of block sizes greater than 32k is not possible (block size must be a correct value)
* FLCC does not add hard coded defaults to generated FLCL commands anymore
* Improve MF-EDZ support for FLAM4 record interface (WIN/UNIX)
** Support usage of the FLAM record interface in EBCDIC
* File creation and modification times are now set correctly
* Improve Windows and UNIX install.txt with first steps after installation
* Suppression of trailing whitespace for record format OPN-TXT is only done if SUPPAD is specified for FIO.REC()
** Add 0x0D (ASCII and EBCDIC) to record delimiter if record format text is used (was suppressed to date)
** Record format text requires well formed text delimiter (dirty delimiter are only supported with format text)
* Improve ASCII EBCDIC separation in character set detection
* Add block size parameter to HASH command (no longer uses hard coded value (1MiB))
* Sort member list of a ZIP archive to ensure ascending offsets (can improve read performance)
* Improved general I/O performance
* Support STREAM (read from STDIN and write to STDOUT) and DUMMY files in FIO.FL4() component
* Use STREAM as default in FLAM command if no file name given to be backwards compatible with the old FLAM4 utility
* Allow empty parameter strings for all FLCL commands ("FLCL XCNV" == COPY "FLCL FLAM" is DECO from STDIN to STDOUT)
* Add CCA/ICSF support on USS (use of hardware cryptography like on z/OS)
* Support for clear key cryptography with CCAPGP and FKMECCA
** Use of ICSF without crypto cards (CC)
** A cryptographic accelerator (CA) for PGP is required
** Supports clear or protected DES, 3DES and AES keys with ICSF
* Move the (CCA-specific) key label template (!FLAMPGP.%u.%t) parameter from PGP to FKM5-CCA
* Default PGP FKM5 function for the non-mainframe world is now PGPRING (PGP key rings) instead of P11
* Add some common use cases as examples at the beginning of the FLCL user manual
* Add default IDENT (owner ("limes")) for logging facilities
* Change all trace messages of FLUC to debug log messages for easier error analysis
* Add NIST self test to crypto kernel
** Runs a self test against test vectors at initialization
** Ensures that all assigned algorithms work correctly
* Fix damaged ZIP files with offsets greater then 4 GB on Windows
* Support "..." or '...' in subsystem allocation parameter (required for LOGR)
* Fix reading of encrypted FLAMFILEs via directory walk
** Processing of the second member emitted a wrong key error because the key value was securely erased from memory too early
** The password or FKME parameter are now only set once per FLAMFILE and not for each member
** Set secure info parameter to member by default if directory walk or member access used
* Windows context menus are now working properly on all supported windows versions
* Add usage reporting to a log stream for z/OS
** The usage reporting can be used to determine cost savings using ZEDC, ZIIP or other co-processors with FLAM
** This feature was implemented for all customers using our programming interfaces to determine the CPU usage for each FLAM component
** See install.txt for z/OS for more information how to configure and evaluate the log stream
** Add new log type USAGE which activates writing of usage log entries also to the normal log (SYSOUT)
* Add new set of environment variables (FL_DEFAULT_PGPRNG/P11/CCAPARA) to define the default FKM5 PGP parameter
* Correct OpenPGP S2K method if hash is smaller than needed key and you have to calculate second hash
* Support system variables for environment variable LANG and ENVID on z/OS (see install.txt)
* Support system variable to control usage of ZEDC in FLAM4 for MODE=ADC (see install.txt)
* Correct AMODE and RMODE in FLMDIR (fix segmentation fault (0C4) in build 15352)
** Default values of linkage editor for AMODE and RMODE are different between z/OS 1.13 and 2.2
* Correct whitespace handling in string converter for numbers
* Correct wrong relative time entries over the command line
- 0000823: [b. Packaging] JAR file for the java byte interface and stream classes are missing in FLAM packages (Tilo Eckert) - resolved. - 0000822: [1.1 FLCL] Add support for system variables on z/OS (Falk Reichbott) - resolved. - 0000826: [1.2 FLCC] Prevent RACF and LOGR messages if no log stream for usage logging defined (Rolf-Dieter Euler) - resolved. - 0000811: [4. Element, Byte, Record or other API] Support of SMF record type 89 for FLAM components (Rolf-Dieter Euler) - resolved. - 0000742: [2.2 Subprogram FLUC (CONV)] Support of PGP key rings in FLAM (Mykhailo Moldavskyy) - resolved. - 0000746: [2.2 Subprogram FLUC (CONV)] Support all asymmetric algorithm of RFC4880 with PKCS#11 (Mykhailo Moldavskyy) - resolved. - 0000740: [2.2 Subprogram FLUC (CONV)] Support all specified symmetric algorithmns of RFC4880 (OpenPGP) (Mykhailo Moldavskyy) - resolved. - 0000821: [2.2 Subprogram FLUC (CONV)] Support DUMMY at read and write of FLAMFILEs on UNIX and WINDOWS (Ulrich Schwab) - resolved. - 0000819: [1.1 FLCL] Support STREAMing for FLAMFILEs in FLCL (Falk Reichbott) - resolved. - 0000658: [2.2 Subprogram FLUC (CONV)] Write converted data of FLUC direct with SSH to remote system (Tilo Eckert) - resolved. - 0000134: [2.2 Subprogram FLUC (CONV)] Use FLAM4 system header for normal FL5-IO (Falk Reichbott) - resolved.
ATTENTION: Non backward-compatible changes in version 5.1.12
Starting with version 5.1.12, the object FL4 of the new overlay ARCHIVE
(archive.fl4()) replaces the former FLAM4NDC switch, the FLAM4 object of
the COMPRESS overlay (comp.flam4()) and the F4PWD/KME objects of the
ENCRYPT overlay (encr.f4pwd() or encr.f4kme()) in the writing command
part. If you use one of these possibilities to create FLAMFILEs with the
CONV command of FLCL, you must adjust the syntax of the WRITE clause of
the CONV command using the archive.fl4() object.
When using cnv.rec(method=DLM) in the reading portion of an XCNV command
or cnv.blk(method=DLM) in the writing part, binary record delimiters are
parsed or written, respectively. When writing via cnv.blk(method=DLM),
the single byte text delimiter remains the default, but when reading via
cnv.rec(method=DLM), a binary record delimiter must be specified
(through the recdlm parameter), otherwise an error will occur. In order
to parse for the default text delimiter, the keyword SYSTEM or the new
TXT keyword can be used.
The keyword FLAMFILE of the DECODE level parameter in CONV commands was
changed to ARCHIVE. If you are using commands like
read.bin(decode=FLAMFILE) or read.rec(decode=FLAMFILE), you must replace
the FLAMFILE keyword with ARCHIVE.
* Support for ZIP archives in CONV commands
** Transparent reading of archives (no new parameter required)
** To write ZIP files, a new overlay (archive.zip/fl4()) was implemented (replaces FLAM4NDC, F4PWD/F4KME and FLAM4)
** Support SupPad and RecDlm in FMT/CNV.REC() for reading and in FMT/CNV.BLK() for writing
*** Parsing of binary record delimiter as formatting or conversion step
*** Additional job step using FIO.REC() is no longer required for this kind of conversion
** Support of compression and encryption with block oriented formats in write.record()
*** Can be used to archive host datasets in BZIP, GZIP, PGP files and/or ZIP archives with FLAM
*** Block conversion with a length field for each record is done by default
*** Record attributes can be put between the length field and the record data (PRNCONTROL=RETAIN)
*** The file attributes (DCB) are stored in GZIP/ZIP member header private extra field or in the ASCII armor header
*** Default behavior, if the decode switch is activated, is decoding of FLAMFILES, ZIP archives, PGP/GZIP/BZIP2/XZ files
* Fix damaged ZIP archives based on wrong offsets returned from z/OS runtime environment
** In rare cases the z/OS runtime environment calculates wrong offsets and writes wrong data
* Support for wildcards (directory walk) for PDS(E), ZIP and FLAM4 archives if a static DD name allocation is used for the library
** For example: file='DD:INPUT(*EGB*)/?*HOGO*' or name='DD:INPUT(*EGB*)' member='*HOGO*'
** The member name of the PDS(E) can also be fully qualified (name='DD:INPUT(EGBERT)')
** If you provide only a DD name, then a directory walk will be done if a PDS(E) is allocated
* flmopn.dll (copy of fl4rec.dll) to Windows and Linux packaging to simplify the use of the Microfocus support
* Only add ".txt" as extension in FMT.TXT() when a text delimiter was added to the data
** If record or binary formatting is done, then the file name is not changed
* Reduced call stack in I/O modules to improve performance
* Return an error when writing data to a directory of a PDS(E)
* Fix return code handling of flmset/flmqry in Microfocus support
* Archive handles (ZIP/FL4) have not been closed and freed properly in some cases if opening the archive failed
** This problem existed since version 5.1.5 in the batch utility (FLCL) and subprograms (FLUCUP)
** Byte, stream and record interfaces are not affected
* Add paragraph to documentation that a write of ZIP archive to PO member on z/OS is not possible
** The runtime environment does not support seek() and tell() functions for PDS(E) members when writing
** Existing ZIP archives can be copied to a PDS(E) member and read properly, but they cannot be written directly
* Improve record length field detection (support binary delimiter and record attributes)
* Correct slots for RRDS number when reading (fix of issue 808) of FLAMFILE with FLUC
* Correct gaps for slots numbers for RRDS when writing FLAMFILES with FLUC
* Map correct default record format for ESDS, KSDS, RRDS when writing FLAMFILEs with FLUC
* Make the charset variable for local file headers in ZIP archives (was fixed to UTF-8)
* Improve performance when writing ZIP archive (reduce number of seek operations)
* Correct default file name for ZIP archives (hugo.gz --> hugo.gz.zip)
* Fix segmentation fault in FIO.FL4() if default archive names used in directory walk
* Correct statistics output of I/O modules if directory walk is used
* Keep I/O handles open when writing to multiple archive files in parallel using the directory walk
** Reduce close and reopen of files when writing (improves performance, minimizes logging)
** Appending data to archives that have been written to before now always works as expected
* Remove empty file attributes from statistics output
* Correct compression ratio and efficiency in statistics output
* Add new log type SUMMARY for archives
* Add all relevant file attributes (record format, record length, ...) to ARMOR header
** This can be used to archive host datasets in ASCII armored PGP/BZIP2/XZ/GZIP files
** Simplifies the exchange of host datasets with PGP
*** For fixed length record datasets only "RecordLength: value" must be added to the armor header
*** For variable length record datasets "BinaryDelimiter: hexvalue" or "LengthField: keyword" must be added
* Add binary delimiter support to private member header of GZIP and ZIP files
** Gives the same behavior which was implemented with the armor header above
* Fix segmentation fault on FLUC record interface if empty statistic buffer provided at FCRCLS
- 0000808: [2.2 Subprogram FLUC (CONV)] RRDS slot numbers lowered by 1 (Falk Reichbott) - resolved. - 0000807: [1.1 FLCL] FLUC don't write into a pds without member name (Falk Reichbott) - resolved. - 0000795: [2.2 Subprogram FLUC (CONV)] Support ZIP archives in CONV command (Falk Reichbott) - resolved. - 0000806: [2.2 Subprogram FLUC (CONV)] Directory walk for member does not work if a static DD name allocation done for PDS(E), ZIP or FLAM-Archive (Falk Reichbott) - resolved.
* Add support for ZIP archives of version 2.0 (no encryption)
** Add redundancy detection (automatically skips compression if no redundancy is found)
** Use FIO.ZIP() in conjunction with CNV.ZIP() via XCNV command to read and write compressed ZIP archives
*** Without CNV.ZIP() files are stored uncompressed
** The format version required to read the written ZIP archives is 2.0
** DEFLATE and BZIP2 compression are supported
** Encrypted ZIP archives are currently not supported (workaround: store encrypted PGP files in the ZIP archive)
** UTF-8 and ZIP64 are supported to allow archives larger than 4 GiB
** The extra field is used to store important file attributes (DCB parameter) for ZIP archives and GZIP files
* Add FLAM4 record interface support for Microfocus EDZ on Windows and Linux
** Develop host-compliant COBOL programs using FLAM4 record interface on Windows and Linux
** Supports only sequential I/O (no insert, delete, update, search)
** Usable also with FLCL utility and subprogram for FLAM4FILEs known by MF-EDZ catalog
** Works like data set and DD name support under USS for MVS ('DD:name' or //'data.set.name')
** Add new FL4REC documentation in DOCZIP and DOCPDF (valid for all platforms)
* New formatting method format.element() available in FLUC byte and record interface
** Allows reading and writing of raw FLAM5 elements
** Provides the matrix type, element type, attributes, element data and hash values for each element
** Useful for example to get access to parsed XML elements
** New sample programs in C (SCFCBELM, SCFCBDOM) and COBOL (SOFCRXML)
** See byte/record interface API docu for details
** Add Cobol sample SOFCREXV for it
* Add element conversion capabilities to byte and record interface if format.element() is used
** Several conversions can be applied to elements read or written elements
** Conversion modules can be opened and then passed to the read and write functions to apply the conversion
** Usage example: Convert XML elements to integers / floating point numbers or collapse strings (remove whitespace)
* XML data elements read via the element formatter now contain larger data chunks than before (i.e. data in tag body is split less often)
* DTD XML elements and DEFAULT elements can now be suppressed when reading by using the NODTD or NODFLT switch, respectively
* New WllFmd switch when writing XML to verify that the XML document output is well-formed XML
** Useful if format.element() is used on byte or record interfaces
* Add FCRLOC to record interface (fcbread() with size==-1)
** Reads records or elements and returns a pointer to a record or to an element structure and its length
** Improves performance and makes variable length data more simple to handle
* Add fcbprev() function to FLUC byte and FCRPRV to record interface
** Sets the read position to the previous record or element
** Can be used, for example, to simplify scanning of elements or to place back the last read byte/character (implemented as fcbungetc())
* Add FCRFLU to FLUC record interface to flush buffered content
* Add fcbgetc(), ungetc(), putc(), gets(), puts(), clearerr() functions to FLUC byte interface
** Most common functions known from the C runtime are now available
* Add matrix type to state string of byte and record interface
* Add data type and format to state string in byte and record interface (only returned when reading, not required when writing)
* Correct syntax for input and output file string for FLUC stream, byte and record interface
* Correct manpages and help information for FLUC stream, byte and record interface
* Correct logic of the DECODE parameter of read.char() and read.text()
* Add log message to FLUC about GZIP/ZEDC on open platforms
* Improve performance and memory consumption of the PGP component
* Add log format MINIMAL which prints log messages without any prefix
* Add log type OUTPUT to separate output values (for example checksums)
* Add HEX/BIN formats and CUT parameter to get sub-hash values to CNV.HSH() component
* Unmask last parameter in load modules to prevent wrong length calculation and other possible mistakes
* Significantly improved API documentation in Doxygen HTML pages
** Improved the general structure and completeness of descriptions
** Better layout
** New appendix for format and state strings of FLUC byte and record interface
* Add support for reason codes to FLUC subprogram interface (FLUCUP)
* Add user table for character conversion to read specifications of CONV command to support subset validation at read
* Add file name mapping for GZIP and OpenPGP header
* Make ASCII-Armor header available in all cases when writing
* Fix segmentation fault if only an empty record was written via record and byte interface before closing the file
* Support member name as explicit parameter in write.flam()
* Command line processor (CLP) accepts integers as floats if a float is expected
* Add LBI Support (tapes with large blocks) in FLAM4 for z/OS
* Support HEADER=NO in FLAM4-Subsystem (no member headers for files in FLAM4 archive)
* Increase internal record length by delimiter length in FIO.REC() when writing to prevent record length errors
* Fix wrong CCSID and CHRSET in FIO.FL4() if the old translate parameter was used
* Ensure that SKPNOB works for CNV.BAS() without auto detection and format error is given
* Add generated copy books for FLUC record (SOFLCREC) and subprogram interfaces (SOFLUCUP) to SRCLIB
* Length format specification with system endianess now works correctly (was always little endian)
* Fix FLAM4 error messages for system errors
* Significantly reduce memory consumption of the CLEP
* GRAVE ACCENT (` - U-0x0060) to enclose strings is now supported by the command line processor
** Useful on some platforms to specify strings in strings, e.g. for FKM5 parameters
* Correct dynamic allocation of UNIX pathnames on z/OS in FIO.FL4()
** FLAM4FILEs are now allocated correctly in USS if FLCL is used
* CNV.PGP() now returns a format error if an internal buffer too small for remaining rest data
** This can only occur if something is wrong with the PGP file format
* Correct use of CSFKDSL in FKM5-PGPCCA to provide PKDS only solution without key store parameter
** Requires z/OS 2.2 or update to ICSF FMID HCR77B0
** Resolves the log issue because a direct VSAM locate against the PKDS beside ICSF is not longer required
** With this solution the FKM5 object is not longer required if you write PGP files with ICSF
* Fix error (ISSUE 797) in PGP module which result in raw cases in a rest buffer overflow or format error
* Use canonical path for member names at write of FLAM4FILEs with FLUC
* Support of slot numbers (RRDS), ASA and machine control character at read with record conversion and formatting
* Support write of attributes in front of records at block conversion and formatting
* Support attributes (ASA, MCC, ...) if text formatting used at write with method record
* Info function detects now empty files (not mark as binary file anymore)
* Integrate FLVIEW into FLTOC ISPF panels of FLAM4 on z/OS (supports V character for it)
* Implement workaround for ZEDC firmware bug (PMR) for FLAM4FILEs
** The ZEDC need few bytes more output buffer for decompression than input was given at compression
** FLAM4 provides now 4 byte more output buffer to solve this issue
** IBM plan to fix this bug within the next firmware version of the card
** Our implementation is now independent of the ZEDC firmware version
* Improve detection of files smaller than the given block size
** In raw cases the PGP module gives an error to increase the block size
** But if the whole file smaller than this block size this makes no sense
** In this case a format error is now returned to better support the transparent read
* Use FMT.BIN() as default at write if FMT.REC() used at read to keep the records
* Use FMT.BIN() as default at write if FMT.BLK() used at read to keep the blocks
* Fix segmentation fault in CNV.GZP/BZ2/LXZ() if memory allocation failed
* Fix error at reorganization of ZIP files if a static allocation for the ZIP file used on z/OS
- 0000799: [2.2 Subprogram FLUC (CONV)] Support retaining ASA and machine control character in ZIP archives (Falk Reichbott) - resolved. - 0000797: [2.2 Subprogram FLUC (CONV)] Read of undefined datasets in FIO.BLK() without FCRBLK does not work with CNV.PGP() (Mykhailo Moldavskyy) - resolved. - 0000701: [2.2 Subprogram FLUC (CONV)] Add support for ZIP archives (Falk Reichbott) - resolved. - 0000203: [2.1 Subprogram FLAM (COMP/DECO)] Support of zEDC Express (Data Compression Acceleration) for FLAM4 ADC on SystemZ (Falk Reichbott) - resolved. - 0000626: [y. General] Support of FL4REC on Windows under EDz from Micro Focus (Falk Reichbott) - resolved. - 0000779: [2.2 Subprogram FLUC (CONV)] Skip DEFAULT and DTD elements at read.xml() from element list (Falk Reichbott) - resolved. - 0000785: [1.1 FLCL] Converting input file names to gzip-names on output (Falk Reichbott) - resolved. - 0000780: [2.2 Subprogram FLUC (CONV)] Add parse parameter to write.xml() to ensure that well formed XML is written (Tilo Eckert) - resolved. - 0000781: [4. Element, Byte, Record or other API] Support individual conversion of element data on byte and record interface (Falk Reichbott) - resolved. - 0000778: [4. Element, Byte, Record or other API] Support reason codes in FLUC subprogram interfaces (Falk Reichbott) - resolved. - 0000775: [2.2 Subprogram FLUC (CONV)] new entry FCRLOC (Falk Reichbott) - resolved. - 0000634: [2.2 Subprogram FLUC (CONV)] Add format.element() to byte and record interface (Falk Reichbott) - resolved. - 0000771: [2.2 Subprogram FLUC (CONV)] Extend hash functions with new formats, charsets and hashlengths (Mykhailo Moldavskyy) - resolved.
* Fix remaining rest error on rare occasions at decryption of PGP files
* Change default behavior for base decoding of CHAR and TEXT data after decompression in read.char/text()
** CMPDEC is now the default and only if the result is XML the base decoding after decompression is done automatically
** If the result of the decoding is only CHAR or TEXT then the base decoding is only done if DECODE=ALWAYS defined (old default)
** The base decoding of text can result in valid but nonsensical text (missing uniqueness)
** The default behavior is not automatic decoding for character and text files anymore if only CHAR or TEXT detected afterwards
** If you change the property FLCL.CONV.READ.CHAR/TEXT.DECODE to "ALWAYS" the old behavior can be configured as default
* Map PGP key flags to usage and vice versa to support better the content of PGP certificates
* Support cross signing with inner self signing as default to get future keys imported by GnuPG
* Add check of valid period between primary and subkey, print warning if this is different (PGPCCA)
* Change default PGP symmetrical algorithm from AES128 to AES256
* Set log type for an FIO-ERROR to a warning if only a record truncation the reason
* Ignore FIO-ERROR if VSAM_POINT fails with 0x00080010 (record not found)
* Improve error message in FIO-ERROR handler in case of no ABEND
* Don't do record length field detection in binary data if it was not requested
** If the record length was defined at read.binary() then also record length field detection was done
* Eliminate unnecessary record conversions at read.binary() for fix length formats
* Add missing new lines to PGP error messages
* Support of one-pass signatures for PGP message files
** Automatic verification of signatures at read (can be disabled)
** Generation of signatures on request (given sign id) at write
* Correct handling and verification of valid days (expiration date) for PGP keys
* Support handling of preferred stuff for PGP key certificates
* Support more than one PGP key set per user ID (full key life cycle management)
** Select the youngest of active keys for encryption and signing
** Export and deletion of keys based on a current or specified timestamp
** Generate key sets for a certain activation date and for given number of valid days
** Manage activation time and valid days (expiration date) correctly
** Print info message to the log to warn before a key will expire
*** The default amount of days is 90 for this warning
*** This can be changed via environment variable FL_KEY_EXPIRE_WARN_DAYS
*** A value of 0 disables this log entry
** The key list function shows activation and expiration dates for PGP keys
*** The indicators below are used in addition:
**** ACT for an active (valid) PGP key
**** WRN for a PGP key which will expire in the next days (warning)
**** EXP for an expired PGP key (not usable for encryption and signing)
**** INA for a inactive PGP key (activation date in the future)
**** KID for a PGP key id based key
**** CCA for all other keys
* Log if hardware acceleration (ZEDC) is used for PGP compression
* Support verification of the primary key ID at PGP key import
** Verification of the imported key ID via the log entry is not longer required
* Write correct padding for FIX record datasets ourselves on mainframes
** Ensures that the correct padding character is used for MVS datasets
** Reset padding character after change of data representation (chrset or binary)
** Add CHRSET to inverse function to ensure correct padding with FLVEDIT
* Correct PGP key import function to support FB80 datasets if no delimiter was found
* Reduce primary and secondary space if dynamic allocation failed with DISP=NEW
** get read of small members from big PDS(E) up and running
** required that FLVIEW and FLVEDIT works in these cases
* Add missing FKM5 parameter at auto decryption (CNV.CRY())
** Transparent read of public key encrypted PGP files now works without any parameter
* Correct key store handling for ICSF (no enforcement to provide a keystore file)
** make CSFKDSL support available (was implemented but not usable)
** don't request a key store file at read (decryption based on key IDs will work automatically)
* Support key label mapping in CCA key names (get replacements in user IDs up and running at import and generate)
** User IDs like '^.HUGO' now work at key generate and private key import for CCA devices
* Fix missing key id printout in key delete function for PGP
* Change when licensing checking is done to avoid auto detection problems when feature code are disabled
** Required update for all customers without the CNV.PGP feature
* Correct and improve some error messages and add more trace messages
* Write PGP level name and CPU usage per level to statistics output
** Show the CPU usage for encryption, compression and literal package building
* The LEVEL parameter to limit PGP decryptions now works correctly
* Add new sample program FLPGPKEY to demonstrate usage of subprogram FLUCKEY
* FLCC commander can show the keyword abbreviations in different ways
* FLCC commander supports JCL templates for command generation
* FLCC commander provides commands in mono space font for editing
* Use uniform time format (YYYY-MM-DD HH:MM:SS) in printouts
* Add switch ALL to key list.pgp() function to show only PGP keys as default
* Support shortened time entries (0t2015, 0t2015/04/01, 0t2015/04/02.23:13)
* Fix wrong hour at time entry if daylight saving time used
* Support locale independent interpretation of floating point values in CLEP
- 0000769: [2.2 Subprogram FLUC (CONV)] Correcrt cross signing of pgp key files (Mykhailo Moldavskyy) - resolved. - 0000772: [2.2 Subprogram FLUC (CONV)] Auto base decoding at read.char/text() result sometimes in errors or demaged data (Falk Reichbott) - resolved. - 0000763: [5. FKME/FKM5] If more then one key PGP availabel, choose the youngest of the actives one (Falk Reichbott) - resolved. - 0000741: [2.2 Subprogram FLUC (CONV)] Support of signatures in PGP files (Mykhailo Moldavskyy) - resolved. - 0000766: [1.2 FLCC] Support JCL template for FLCL command generation (Ulrich Schwab) - resolved. - 0000760: [2.2 Subprogram FLUC (CONV)] Show remaining valid days of a PGP key and give a warning some times before expire (Mykhailo Moldavskyy) - resolved. - 0000761: [2.2 Subprogram FLUC (CONV)] Show zEDC usge in PGP compression layer (Mykhailo Moldavskyy) - resolved. - 0000758: [2.2 Subprogram FLUC (CONV)] Add parameter "KEYID" to flcl key import, for verify this during import (Mykhailo Moldavskyy) - resolved. - 0000762: [6. Access Control, Logging and Errorhandling] If a feature disable but used in auto detection, then read.auto/text/char/binary don't work anymore (Falk Reichbott) - resolved. - 0000754: [2.2 Subprogram FLUC (CONV)] Text files some times padded with 0x00 instead of blank (0x40) (Falk Reichbott) - resolved. - 0000764: [5. FKME/FKM5] The key id label print out is missing if FLCL KEY DELETE function used (Mykhailo Moldavskyy) - resolved. - 0000755: [2.2 Subprogram FLUC (CONV)] If the owner (^) used in a PGP userid, then the key label is not valid. (Falk Reichbott) - resolved. - 0000723: [2.2 Subprogram FLUC (CONV)] If a small member of a big PO converted, then the file space for the target can be very hugh (Falk Reichbott) - resolved.
* Support filename type for strings to read passwords from files (PASSWORD=f'pwdfile.txt')
* First version of OpenPGP support (AES+RSA+SHAx with PKCS#11 or CCA/ICSF)
** Transparent reading and writing of OpenPGP files compliant to RFC4880 with CONV and XCNV commands
** Add new manual for FLAM key management extension in version 5 (FKM5BOOK) which describes the CCA/ICSF and PKCS#11 implementation
** Add new FLCL command KEY to manage PGP keys (Generate, Export, Import, List, Delete) for PKCS#11 or CCA-based HSMs
* First version of FLCC-Commander for Windows and Linux (GUI to build commands, manage property files, interactive manual)
* If environment variable substitutions are used (<VARNAME>) and the corresponding environment variable is not defined, the variable string remains
** i.e. is not replaced by an empty string, except if the environment variable is set to only whitespace characters
** Trailing whitespace is removed from environment variables
* Support of fix length record data exchange over block oriented systems (GZIP, OpenPGP, ...)
** Add RECLENGTH parameter to read.binary() and read.char() to support splitting binary/character data into fixed length records
** Has data after binary I/O, decoding, decryption or decompression a fix length format the records can build if the correct record length defined
* Add parameter TARGET to CNV.BAS() which limits the valid formats after base decoding (e.g. make base decoding only if the result is a PGP message)
* Change default owner ID from 'de.limes' to 'limes' to be in sync with key label template management
* Support for setting OWNERID and ENVID as environment variables in read and write operations of the CONV command
* Support for 4 byte length fields in write.text() as new methods
* Support for a predefined record length for ASCII-armor detection if no delimiter or length table available
** Can be freely defined with XCNV command
** Is hard-coded to 80 if CONV command is used
* Add documentation for the FLUC byte interface in Java as FLCBYTJS.zip/DOCZIP(FLCBYTJS) to each package
* Add new environment variable to disable hardware acceleration of cryptographic operations (FL_DEACTIVATE_HW_CRYPTO_ACCELERATION)
* Fix error in password encryption of FLAM (rare segmentation faults with x86 hardware acceleration)
* Fix wrong FLAM4 compression mode for NDC on z/OS (z/OS use 5 instead of 11)
* Don't print empty file log entry at CNVBLK/REC (use internal FMTTXT without data) anymore
* Don't switch to binary data handling at text formatting if character conversion was successful in front of it
* Improve statistics output (print used method)
* Add more samples to FLCL book
* Fix CLEP lexical error message "Character ('%c') not valid"
* Correct build of LIBFKME for FKMEFILE (entry was not exported for DLL usage)
* Fix buffer overflow in FMTXML when writing CDATA elements
* Support INDSIZ, INDCHR and NOCMNT parameters for XML formatting at write.record/flam4/text() if read.auto() is used
* Transparent read of FLAMFILEs supports now wrong record length in member headers
* Inverse command files works if a hash '#' character is in a variable (file name) of the CONV command list
- 0000750: [2.2 Subprogram FLUC (CONV)] Support longer records at read of FLAMFILEs as defined as record length in the member header (Falk Reichbott) - resolved. - 0000739: [2.2 Subprogram FLUC (CONV)] Support passphrase encrypted PGP files (Mykhailo Moldavskyy) - resolved. - 0000730: [1.1 FLCL] XML (and other) parameters in write.*() are ignored when using read.auto() (Falk Reichbott) - resolved. - 0000724: [2.2 Subprogram FLUC (CONV)] Support multiple receiver for PGP (Mykhailo Moldavskyy) - resolved. - 0000075: [2.2 Subprogram FLUC (CONV)] Add OpenPGP support to data conversions (Falk Reichbott) - resolved. - 0000718: [2.2 Subprogram FLUC (CONV)] Support 4 byte length field formats as write.text method (Falk Reichbott) - resolved.
* Add ASCII armor (also supports EBCDIC) with checksum support to base encoding component
** This is in preparation for the upcoming OpenPGP support in FLAM (RFC4880)
** In addition to Radix-64 (RFC4880), encoding with Base16 (+CRC32 checksum) and Base32 (+CRC40 checksum) is supported
** The armor header and trailer can be used for any kind of data and encoding schemas
** Base decoding now also supports concatenated encoded streams
* Fix incorrect statistics of base encoding component
* Support 4 byte record length field detection and handling
** Host files from ZIP archives can contain length fields (L4I), which must be used to build records when reading block-oriented
** If a USS record file was allocated with FILEDATA=BINARY, then the length fields are used to build records when reading block-oriented
** If record format VAR is used on Windows or Unix systems, then the length fields are determined automatically when reading block-oriented
** Add additional variants of the host record length formats (LLxx) for record I/O (incompatible change, see warning above)
** Add support for adding length fields before compression/encryption/encoding when using write.binary() and write.char()
* Increase default block size from 16k to 64k for better detection results, also used for record I/O to build blocks
** This change also causes at block oriented I/O, that files written on z/OS are always allocated with the system default block size
because the internal default block size is always out of the valid range and in this case 0 is used at dynamic allocation
* Increase default block size only if no block size is defined via the command string
* Provide CNV.ZIP() on input side of XCNV command for auto decompression in conjunction with FIO.ZIP (not yet available)
* Support for special condition codes
** 112 - Try again with an ASCII-based CCSID after incorrect UTF-8 detection at read
* Add detection of file types that are based on ZIP archives (JAR, DOCX, ODT, APK, ...)
* Add new switch RENEW to FALLOC object to enforce removal of an existing file before allocating it for writing
* Add new DIFF command for logical neutral comparison of two data sources (works like reading two sources with conv.read and comparing the two results)
* Use 32752 as default record length if no member header is available in a FLAMFILE (makes FLVIEW work for FLAMFILEs without headers)
* Improve info function if no member header is available in a FLAM4FILE
* Improve error trace if error EILSEQ (illegal sequence) is encountered during character conversion
* Log transliterations to the report file in any case at character conversion
* Indicate substitutions with nothing as IGN (ignore) in the character conversion report file
* Eliminate the redundant entry for stop (STP) at EILSEQ (illegal sequence) in character conversion report file
* Clarify in documentation that the counting of position parameters starts with 0 for character conversion errors and report entries
* Reduce UTF-8 detection probability to 50% if all analyzed characters are in the 7-bit ASCII range (0-127)
* Use message "byte order mark (BOM)" in character conversion report file if BOM ignored
* Use AES hardware instructions (if available) to improve AES performance on x86 based systems
* If not already defined, the current owner is now available as environment variable OWNERID, e.g. to use it in filenames (~/<OWNERID>.dat)
* Activate record oriented write of blocks, if records without delimiter defined in this blocks
* Remove redundant log messages about AES calculation in hardware or software
* Improve error trace if writing of a record failed
* Allow user to turn off all logs.
- 0000566: [2.2 Subprogram FLUC (CONV)] Support RADIX and checksum for base encoding (Tilo Eckert) - resolved. - 0000712: [1.1 FLCL] Support of special condition codes (Falk Reichbott) - resolved. - 0000713: [2.2 Subprogram FLUC (CONV)] Add a switch to FALLOC object at write, which enforce a disposition NEW allocation, by deleting the old file (Falk Reichbott) - resolved. - 0000711: [2.2 Subprogram FLUC (CONV)] Add utility command for a logical compare of 2 files (Falk Reichbott) - resolved. - 0000705: [2.2 Subprogram FLUC (CONV)] Write also character translitterations to report file (Mykhailo Moldavskyy) - resolved. - 0000710: [2.2 Subprogram FLUC (CONV)] FLVIEW don't work for some FLAMFILEs (Falk Reichbott) - resolved.
* Fix major error in PDS(E) directory resolution
** Sometimes not all members of a PO library were found
** In rare cases, reading of a directory is terminated too soon
* Use catalog search interface for directory walk on ZOS
** Support wildcards at the beginning of the high level qualifier '**(IGGCSI00)'
** Distinguish between files in DASD (only included at default) or TAPE (only included if switch defined)
** Avoid opening of each file to figure out if it is a PDS
** Support of aliases to data sets like symbolic links on other platforms
** List all files from master, user and disk catalogs
** Give a warning for each file which is not usable
* Write a given delimiter behind each record on record I/O
* Add disk number to XML listing of INFO function for ZIP file members
- 0000703: [2.2 Subprogram FLUC (CONV)] Write delimiter after each record if defined (Falk Reichbott) - resolved. - 0000700: [2.2 Subprogram FLUC (CONV)] Wild cards in PO data sets don't work correctly (Falk Reichbott) - resolved.
* Add first version of ZIP archive support (info get.file list now ZIP file directory content)
* Improve significant I/O performance (factor 2 EXCP, factor 3 CPU for all I/O modules)) on z/OS
* Improve ABEND handling for I/O errors on z/OS (requires at minimum z/OS v1r13)
* Use catalog search interface to determine used file size, block size (=CISIZE) for VSAM datasets and other attributes
* Add zEDC-Support for GZIP-Files and GZIP compression suites (not yet for FLAM4FILEs (ADC))
* Add definition of maximal condition code (MAXCC) for command execution over FLCL
* FLVEDIT works now also for bigger files (don't define record length (32756) anymore)
* Add FKMEFILE to libfkme (read key value from file, to prevent logging of passwords)
* Improve performance at property file parsing (undefined properties are now line comments)
* Fix error at DirWalk in error handling if DirFile failed with an error
* Fix few little errors (wrong source string) in syntax error print out of CLEP
* Change minimum architecture from 2064-100 (z900) to 2097-xxx (IBM System z10® EC)
* Change optimized architecture from 2817/8-xxx (z196/z114)to 2827-xxx (IBM zEnterprise EC12)
* Correct CPU and RUN time values in statistic output of FLUC
* Correct error handling if I/O error occurred at read
* Correct file name if path name allocation used for FLAMFILEs in FLUC (use SCV99(RETRIEVE))
* Correct data type in FMTTXT from BIN to CHR if CHR detected and formatting method BIN is used
* Add new switches to read.text() to enforce an error if binary or character data without delimiter detected
* Change XML printouts of INFO function from value based list to an attribute based list
* Correct printouts of info function for GZIP, BZIP and XZ files
- 0000692: [e. Configuration] Add MAXCC parameter to overrule condition codes of FLCL (Falk Reichbott) - resolved.
* First version with directory support
- Process multiple files at once using wildcard patterns
- Support of wildcards at read, and directory generation at write
- Support for searching inside FLAMFILEs containing multiple files using wildcards
- Support of complex filename mapping based on patterns (e.g. converting host dataset names to Unix or Windows paths)
- Support of index ranges for member access in FLAMFILEs ('./flamfile.adc/#3,5-8,10')
- Add 'Dir' object to define parameters for file and member selection (recursive, hidden, link, archive, mount, ...)
- Add 'Remain' parameter for all kind of write operation to prevent overwrite of existing files
- Add 'NoPath' parameter to write.flam() to eliminate the path from member names
* Uses a new version of CLEP (Attention: condition codes have been changed with this version)
- Change to new condition codes (separation of warnings and error in RUN function)
- Add support of reason codes
- New appendix for return and reason codes in FLCL manual (supported with 'GENDOCU')
- New built-in function 'ERRORS' to list all return and reason codes and corresponding help messages
- Support of different EBCDIC code pages for command interpretation of critical punctuation characters ($!~#{|}[\]@_^)
- Correct error messages (... longer then n ...) for 0-terminated and hexadecimal strings
* Add new FLCL command XCHK
- full XCNV support for INPUT, without OUTPUT overhead
- useful for checking or analyzing input data
* Add new FLCL command HASH
- simplification for hash calculation (like XCHK, without write overhead)
- works like and is compatible with sha1sum and md5sum utilities on Unix
* Add new FLCL command UTIL
- provides simple functions like listing and deleting files based on wildcards
- can be used for example to find the PO library for a certain member
- or to remove all kind of datasets or files based on wildcards
* Add parameter 'INVERSE' to 'CONV' command to build the inverse XCNV command
- This is used by the new ISPF line command FLVEDIT to edit UTF/XML/GZIP and other files under ISPF as EBCDIC records
- flcl conv read.auto(...) write.record(...) inverse=inverse.par <-> flcl xcnv=inverse.par
* Add support for global parameter and property files in FLUCUP
- r=flucconv("=para.txt","=prop.txt",NULL,NULL)
* Support tilde ('~') on ZOS as abbreviation for <SYSUID>
* Add support of output and trace files in FLM4UP
* Log messages (if set) as info after successful call of FKME
* Correct log messages of FIOFL4 component
* Use own log type for progress bar (no default anymore)
* Fix 'remove' flag for files on ZOS
* Increase default primary space from 30 to 600, secondary from 900 to 1200 tracks and directory block from 10 to 45
* Use name of original file as default member name for archives (FLAMFILEs)
* Some performance improvements of all components
* Extend libfkme with FKMESYM (FIN/PIN PCIDSS) for CCA and PKCS11
* Add info, help and documentation for libfkme over FLCL info command
* Add generated FKME user manual to installation package
* Fix wrong mapping of MCC print control at write of FLAMFILES with FLUC
* Fix wrong mapping of record formats FS & VS at write of FLAMFILES with FLUC
* Provide FL5 installation packages for zLinux and USS
* Increase space for parameters parsed with FLAMCLP (filenames up to 1023 character)
* Add makefile (Unix, Windows) and build job (mainframes) for sample programs
- Documents how to use, compile and links with our FL5 API's
- 0000088: [2.2 Subprogram FLUC (CONV)] Archive and directory support for original data sources (Falk Reichbott) - resolved. - 0000617: [2.2 Subprogram FLUC (CONV)] CPACF support for the new CNV.HSH component of FLUC (Mykhailo Moldavskyy) - resolved. - 0000619: [2.2 Subprogram FLUC (CONV)] Add a new subprogram for hash/signature generation and verification (Mykhailo Moldavskyy) - resolved. - 0000627: [2.2 Subprogram FLUC (CONV)] Ignore comments in XML files at read and or at write (Tilo Eckert) - resolved. - 0000637: [2.2 Subprogram FLUC (CONV)] File name with '+' can not be read. (Falk Reichbott) - resolved. - 0000642: [2.2 Subprogram FLUC (CONV)] Add INDSIZ parameter fpr pretty prininting to write.record (CONV) and format.record() (Tilo Eckert) - resolved. - 0000654: [1.1 FLCL] Generate at read reverse command for write to support edit of files (Falk Reichbott) - resolved. - 0000656: [2.2 Subprogram FLUC (CONV)] Segmentation fault if file=DUMMY used in write.record() (Falk Reichbott) - resolved. - 0000657: [1.1 FLCL] Put progress bar under a own log identifier (Falk Reichbott) - resolved. - 0000661: [4. Element, Byte, Record or other API] Support parameter files for flucconv() and other subprograms on command level (Falk Reichbott) - resolved. - 0000662: [2.1 Subprogram FLAM (COMP/DECO)] Support trace file if a environment variable defined (Tilo Eckert) - resolved. - 0000669: [1.1 FLCL] Add new utility command to remove or find files (Falk Reichbott) - resolved.
* Support for '~' as alternative to <HOME> on ASCII systems
* Fix error in CLIST FLDOC on ZOS (absolute filename needed)
* Correct and improve read of blocks on record oriented systems
- Read at least 16KiB or 4 records to ensure correct data detection
* Fix base encoding detection (rejects if non-padding data after end of encoded text is found)
* Add CPACF support for CNVHSH for ZOS, USS and zLinux
* Add more hash calculation methods to CNVHSH (RipeMd128/160, CRC, SHA224/256/384)
* Fix a few issues in FLAM4 kernel of FL5 for distributed platforms (Windows, UNIX)
- Correct and improve detection of FLAM4FILEs
- Add missing deallocation of resources in error situations
- Increase performance of run length compression in CX8/VR8 algorithms
- Correct close at end of a CX7 files
* Don't flush data on write in a case of an error
* Fix missing pointer reset after realloc in character conversion module
* Add key length (KL16/25/32) and ASCII support to FKMECCA2 (z/OS)
* Remove double mapping of file names (don't replace environment variables twice)
* Add NOCMNT switch to read and write XML operations (ignores comments in XML)
* Fix bug when writing XML in pretty printing mode with INDSIZ=0
* Improve performance on Linux systems (switch from GCC to CLANG)
- 0000622: [2.2 Subprogram FLUC (CONV)] Base detector erroneously detects a base encoding in binary data in some cases (Tilo Eckert) - resolved. - 0000623: [d. Installation] Mismatch between Destination Path for Program and Licensefile (Ulrich Schwab) - resolved. - 0000629: [2.2 Subprogram FLUC (CONV)] Support a kind of pretty printing without leading spaces (Tilo Eckert) - resolved.
* Add checksum support to CONV and XCNV commands (GNU and BSD style)
With the new component MD5, SHA1 and SHA256 hashes can be generated
and verified using the FLUC command, APIs and subprograms.This can be
used to check the binary integrity of files. Calculated checksums are
compatible with UNIX utilities like "md5sum" and "sha1sum".
* Support of "IDENTITY" error handling for single to single byte character conversion
* Update UNICODE data to newest version (currently used for case mapping)
* Fix wrong error position at semantic error of CLP (command line parser)
* Add -s option to fliconv sample utility to skip incomplete character at end of file
* Improve progress bar (maximum 72 characters) in fliconv sample utility
* Add more error handling functions to FLUC iconv interface mainly for COBOL and PLI
* Add COBOL sample (SOFLCICV) for using the FLUC iconv interface on mainframes
* Fix error in vertical tab replacement if method original is used (now uses system delimiter for inserted line breaks)
- 0000607: [2.2 Subprogram FLUC (CONV)] Support of RPLHTB/VTB/CTR and SUPTWS also for METHOD=ORIGINAL at write.text() (Falk Reichbott) - resolved. - 0000613: [4. Element, Byte, Record or other API] Add COBOL sample for FLCICV (fliconv) interface (Rolf-Dieter Euler) - resolved. - 0000615: [2.2 Subprogram FLUC (CONV)] Add checksum generation and verification cabebilities for original files (Mykhailo Moldavskyy) - resolved. - 0000616: [6. Access Control, Logging and Errorhandling] Function fliconv_geterrno usable in COBOL/Assembler/... desired (Falk Reichbott) - resolved.
* Provide character conversion module of FLAM as separate library compatible to libiconv (FLCICV)
* Add linux like iconv utility as sample for usage of the new FLCICV library
* Rename sample user tables for character conversion from FLUT* to CCUT*
* Add support for more single byte ASCII and EBCDIC code pages (incl. IBM0273, DIN-66003)
* Support several important subsets as system table (SYSTAB=NPAS/SEPA/DELA/DLAX/...)
* Add more sample user tables (SEPA, 1141, Best Fit Mapping) for character conversion
* Count non-reversible character conversions and log a warning if this count > 0
* Remove padding to correct XML detection in INFO function
* Correct XML detection at auto formatting (mainly for HTML files)
* Improve support of RISC based systems (SPARC, POWER, ITANIUM, ...)
* Add new switch to dump binary data at write.text() for FLVIEW on block oriented platforms
* Improve FLVIEW for block oriented system with dump of binary data
* Add new line command FLTEXT to ISPF (like FLVIEW but with print out for text or dump of binary data)
* The byte interface can be used from Java via Java Native Interface (JNI)
* Support RPLHTB/VTB/CTR and SUPTWS also for method ORIGINAL at write.text()
* Support replacement of form feeds (RPLFFD) also at read.auto()
- 0000593: [2.2 Subprogram FLUC (CONV)] Support of single byte code pages as subsets for UNICODE (Mykhailo Moldavskyy) - resolved. - 0000470: [2.2 Subprogram FLUC (CONV)] Dump of output to file or stream (Tilo Eckert) - resolved. - 0000588: [2.2 Subprogram FLUC (CONV)] Support of string.latin for character conversion (Falk Reichbott) - resolved. - 0000594: [2.2 Subprogram FLUC (CONV)] Support for SEPA subset at character conversion (Mykhailo Moldavskyy) - resolved. - 0000596: [2.2 Subprogram FLUC (CONV)] Support for character set DIN66003 (Mykhailo Moldavskyy) - resolved. - 0000598: [4. Element, Byte, Record or other API] Provide FLAM character conversion modul as separate target (Mykhailo Moldavskyy) - resolved. - 0000600: [2.2 Subprogram FLUC (CONV)] Add sample user table for character conversion which deletes all the combined characters for single byte code pages (Mykhailo Moldavskyy) - resolved. - 0000608: [2.2 Subprogram FLUC (CONV)] Support of replacement of form feeds (RPLFFD) at read.auto() (Falk Reichbott) - resolved.
* Support of String.Latin (Latin characters in UNICODE)
* New XCNV command for user defined conversions
* Support SMPE packaging for ZOS (FMID FLAM5xx + PTF FLIC5xx)
* Added record interface (FCROPN/PUT/GET/CLS) for original data
* Added stream buffer class library for use with C++ I/O stream classes
* Transparent and automated reading of all supported file formats
(including normal data sets, VSAM, FLAMFIELs, compressed, encrypted
and encoded binary, text or XML files)
* Own implementation of character conversion module (incl. subsets)
* Added BASE16/32/64 encoding and decoding for command CONV of FLCL
* Better charset, file format and delimiter detection
* Introduce fl5core, reduces installation package size by factor 10
* Reduced memory usage and increased performance
* Added FLINFO and FLVIEW line commands for ISPF
* Improved error handling, property management and interactive help
* Added sample user table CCUTDEXL.txt (Ä->AE, Ö->OE, Ü->UE, ß->ss) to
change system transliteration for German umlauts and SZ
* Added LANG & PLATFROM parameter to emulate environments at run time
* Unified file name handling (DD:NAME support, data set names)
* Prevention of logging and tracing of passwords and other critical data
* Support execution of commands by owner (flcl owner conv read.text(..))
* Fixes for several multi-threading issues
* Integration of FLAMSUB in FLAM load library
* Added 'remove' and 'flush' parameter for file I/O
* Added man pages for UNIX systems
* Added subsystem support
- 0000417: [4. Element, Byte, Record or other API] Add FLUC stream interface for C++ (Tilo Eckert) - resolved. - 0000550: [2.2 Subprogram FLUC (CONV)] There are a lot of unsupported code pages on AIX (Falk Reichbott) - resolved. - 0000549: [2.2 Subprogram FLUC (CONV)] ICONV on Solaris don't support ISO-Codepages (Falk Reichbott) - resolved. - 0000375: [2.2 Subprogram FLUC (CONV)] Iconv on host does not recognize invalid UTF-8 characters (Falk Reichbott) - resolved. - 0000579: [4. Element, Byte, Record or other API] Last record/block missing after fcbclose if format.text() used (Falk Reichbott) - resolved. - 0000074: [2.2 Subprogram FLUC (CONV)] Add BASE64 and HEX as encodings to conversion layer (Falk Reichbott) - resolved. - 0000183: [2.2 Subprogram FLUC (CONV)] Add vertical tab replacement support to text handling (Falk Reichbott) - resolved. - 0000565: [2.2 Subprogram FLUC (CONV)] Support of an environment variable (PLATFORM) to define default settings for a platform (Falk Reichbott) - resolved. - 0000552: [2.2 Subprogram FLUC (CONV)] Add sample USRTAB for Ä->AE, Ö->OE, Ü->UE and ß->ss transliteration (Falk Reichbott) - resolved. - 0000555: [b. Packaging] Support of SMPE installation of FLAM (Falk Reichbott) - resolved.
- 0000402: [2.2 Subprogram FLUC (CONV)] LANG-Variable auch als Parameter setzen (Falk Reichbott) - resolved. - 0000545: [2.2 Subprogram FLUC (CONV)] Is removed used for read, file flush must be done at write (Falk Reichbott) - resolved. - 0000502: [2.2 Subprogram FLUC (CONV)] Property management are missing in UP interface (Falk Reichbott) - resolved. - 0000457: [2.2 Subprogram FLUC (CONV)] Add FLAM as first encryption method (Falk Reichbott) - resolved. - 0000392: [2.2 Subprogram FLUC (CONV)] Add replacement of control character by one space (Falk Reichbott) - resolved. - 0000489: [2.2 Subprogram FLUC (CONV)] CCSID 'CP1252' don't work (Falk Reichbott) - resolved. - 0000486: [2.2 Subprogram FLUC (CONV)] Support of HOST length field format for write and read of records on open systems (win, unix) (Falk Reichbott) - resolved. - 0000471: [2.2 Subprogram FLUC (CONV)] Transparent read of binary, text and xml files (Falk Reichbott) - resolved. - 0000418: [2.2 Subprogram FLUC (CONV)] read.text() works for record oriented files based on record bounderies if no delimiter found in the data (Falk Reichbott) - resolved. - 0000472: [2.2 Subprogram FLUC (CONV)] If stream used the tranparent read of FLAM- and normal FILES does not work (Falk Reichbott) - resolved. - 0000458: [2.2 Subprogram FLUC (CONV)] read of empy file with asa or maschine control charactres gets length error (Falk Reichbott) - resolved. - 0000437: [2.2 Subprogram FLUC (CONV)] Provide FLCL INFO GET.FILE as ISPF service (Rolf-Dieter Euler) - resolved. - 0000454: [4. Element, Byte, Record or other API] Add statistic to close functions of FLUC byte and record interface (Falk Reichbott) - resolved. - 0000448: [4. Element, Byte, Record or other API] Add get and set function to FLUC byte and record interface to manage the current state (Falk Reichbott) - resolved. - 0000447: [2.2 Subprogram FLUC (CONV)] Add DECO parameter to read.record() to decompress records from flamfiles (Falk Reichbott) - resolved. - 0000445: [2.2 Subprogram FLUC (CONV)] Add subsystem support to read.record() for transparent record IO over different subsystems (Falk Reichbott) - resolved. - 0000440: [4. Element, Byte, Record or other API] Add info function for files to byte and record interface (Falk Reichbott) - resolved. - 0000428: [2.2 Subprogram FLUC (CONV)] CCSID list of INFO command are not complete for z/OS (Falk Reichbott) - resolved. - 0000114: [2.2 Subprogram FLUC (CONV)] Automatic detection of ASCII, EBCDIC and UTF character sets, text and character handling (Mykhailo Moldavskyy) - resolved. - 0000262: [2.2 Subprogram FLUC (CONV)] Support of CCSIDs as 5 digit decimal number (Ulrich Schwab) - resolved. - 0000157: [y. General] Release FLAM5.0.0 (FLUC) (Falk Reichbott) - resolved. - 0000311: [4. Element, Byte, Record or other API] Add FLUCUP as customer API (Mykhailo Moldavskyy) - resolved. - 0000294: [4. Element, Byte, Record or other API] Byte interface for original data (including FLAM4FILEs) (Falk Reichbott) - resolved. - 0000224: [y. General] Support of zEDC on z/OS + USS (Mykhailo Moldavskyy) - closed.
- 0000344: [2.2 Subprogram FLUC (CONV)] Support mapping of DYNALLOC return/info codes to message strings (Mykhailo Moldavskyy) - resolved. - 0000353: [2.2 Subprogram FLUC (CONV)] Add switch to activate NEL as delimiter for ASCII-Charsets. (Falk Reichbott) - resolved. - 0000315: [2.2 Subprogram FLUC (CONV)] Add XML formatting to FLUC conversion (CONV) command (Tilo Eckert) - resolved. - 0000325: [6. Access Control, Logging and Errorhandling] Add a parameter to control the format of info lists. (Falk Reichbott) - resolved. - 0000076: [2.2 Subprogram FLUC (CONV)] Add XML formatting for original data (Tilo Eckert) - resolved. - 0000217: [2.2 Subprogram FLUC (CONV)] Enforce conversion of EBCDIC New Line (NL=0x15) to Line Feed (LF=0x25) to ensure that 0x0A will be used for ASCII/UTF (Falk Reichbott) - resolved. - 0000218: [2.2 Subprogram FLUC (CONV)] Support read and write of FLAM4FILEs on USS (Falk Reichbott) - resolved.
- 0000115: [y. General] Use of enviroment variables to define expansion factors and other constants used for internal calculations (Falk Reichbott) - resolved. - 0000193: [2.2 Subprogram FLUC (CONV)] Support of record mode (stop, cut, wrap) for USS files on MVS (Falk Reichbott) - resolved. - 0000192: [2.2 Subprogram FLUC (CONV)] Binary read of USS files over ZOS-RTE does not work (Falk Reichbott) - resolved. - 0000191: [2.2 Subprogram FLUC (CONV)] File size determination for ZOS (Falk Reichbott) - resolved. - 0000176: [2.2 Subprogram FLUC (CONV)] Support of dummy for file IO (Falk Reichbott) - resolved. - 0000186: [2.2 Subprogram FLUC (CONV)] Add replacement of form feed control character by n new lines at text formatting (Falk Reichbott) - resolved. - 0000168: [y. General] Add new logging types for license stuff (Falk Reichbott) - resolved. - 0000129: [a. Build] Use always fixed size types in format strings (Ulrich Schwab) - resolved. - 0000121: [2.2 Subprogram FLUC (CONV)] Set _ICONV_MODE and _ICONV_TECHNIQUE over FLCL configuration file on z/OS (Ulrich Schwab) - resolved. - 0000002: [2.2 Subprogram FLUC (CONV)] Integrate local independent CASE mapping over tables in CNVCHR (Ulrich Schwab) - resolved. - 0000106: [2.2 Subprogram FLUC (CONV)] Add UTF-16/32 support for character conversion on ZOS (Falk Reichbott) - resolved. - 0000102: [2.2 Subprogram FLUC (CONV)] Default CCSID used for character conversions can be defined as configuration parameter (Falk Reichbott) - resolved. - 0000160: [c. Deployment] Add FLCC and FLCL as Feature to the access module (Mykhailo Moldavskyy) - closed.
