QSMM Changelog

What's new in QSMM 1.16

• Backward-incompatible changes.:
• To improve the robustness of package operation for long event histories, the type of field `fq' of structure `qsmm_cycle_s', fields `fq_goto_min' and `fq_action_min' of structure `qsmm_disasm_desc_s', and field `fq_min' of structures `qsmm_dump_mat_goto_desc_s' and `qsmm_dump_mat_action_desc_s' was changed from `int' to `long'.
• Function `qsmm_get_node_array_prob_out' now will not set the first element of a buffer for retrieved elements of an array to -2 if the array is not found. Instead, the function will raise exception QSMM_ERR_NOTFOUND, to which new entity type QSMM_ENT_ARRAY_PROB is passed.
• Changes for actor:
• Added function `qsmm_get_actor_prob_action', using which a probability of the last signal returned by function `qsmm_get_actor_sig_action' can be obtained. The function operates quickly for small and large actors.
• Added function `qsmm_set_actor_discrete_time' for setting discrete time tracked by an actor. That function can be used when restoring the state of the actor from a file.
• Function `qsmm_actor_time_delta' now supports negative time increments. This feature can be used to restore continuous time when loading a state of the actor from a file. However, the function disallows to change continuous time tracked by the actor to a negative value.
• Added type of relative probability function QSMM_RELPROB_BUILTIN3, which is analogous to type QSMM_RELPROB_BUILTIN2, but is simply the exponent, without addend +1 and divisor 2.
• Added type of relative probability function QSMM_RELPROB_USER2. That type is to be used with a helper function of type `qsmm_relprob_user2_func_t', to which a signal that indicates a cycle direction, condition for the action choice state and statistics for the cycle type are passed.
• Field `sig_cycle_next' was moved to the end of structure `qsmm_state_s', according to the order of fields that depends on their
• types and is accepted in the package source code.
• Changes for multinode model:
• Added support for a mode when the instruction emitting engine (formerly referred to as an engine that produces optimal actions on the basis of current environment states) is represented by a large actor. This mode will be turned on when new field `is_large_opt' of structure `qsmm_desc_s' has a non-zero value. For certainty and full awareness of operating parameters of the engines, it is recommended to explicitly set all those parameters just after creating the model instance.
• Function `qsmm_create' will return QSMM_ERR_INVAL if field `is_large_opt' of structure `qsmm_desc_s' is non-zero and, at the same time, field `use_flat_storage' of that structure is non-zero or field `is_determ_opt' is non-zero or field `dont_use_instr_class_weights' is zero or field `compat' is zero.
• Added field `profile_pool_opt_sz' to structure `qsmm_desc_s', using which the size of the pool of probabilities lists in normal form of an instruction emitting engine represented by a large actor can be specified when creating a multinode model. Function `qsmm_create' will return QSMM_ERR_INVAL if the value of that field is negative.
• A large environment state identification engine can now be used even when field `is_determ_opt' of structure `qsmm_desc_s' is set equal to 0, i.e. when both matrices--the state transiton matrix and the action emission matrix--specify non-deterministic choices.
• Function `qsmm_node_call_default' now loads a default uniform probability profile into a node if the node does not have a probability profile set, the model contains multiple nodes, and one or both engines are represented by large actors. The profile is loaded using an implicitly generated assembler program. If the model has a look-ahead signal segment of positive length, then the loading cannot be performed and QSMM_ERR_UNSUPPLA will be raised.
• The loading of default uniform probability profile is also supported when field `is_determ_opt' of structure `qsmm_desc_s' has a non-zero value and the number of instruction classes in the instruction class set of the node is equal to 1. If the number of instruction classes is greater than 1, then QSMM_ERR_NOPROF will be raised.
• Added functions `qsmm_get_prob_goto' and `qsmm_get_prob_action' that return the probability of the last state transition performed and the probability of the last action produced. The functions can be called during processing an instruction invocation event by the event handler function of instruction meta-class.
• Function `qsmm_set_node_nstate' will now raise QSMM_ERR_NOSYS if the environment state identification engine is represented by a large actor and the model contains a single node with identifier 0. Use function `qsmm_set_nstate_max' to set the number of states of that node.
• Function `qsmm_time_delta' now supports negative time increments. However, the function disallows to change continuous time tracked by each of two actors upon which the multinode model is based to a negative value.
• Added enumeration `qsmm_mat_e' with two elements that specifies the type of a matrix. Element QSMM_MAT_GOTO denotes a state transition matrix and element QSMM_MAT_ACTION denotes an action emission matrix.
• Added element QSMM_ENT_ARRAY_PROB to enumeration `qsmm_ent_e'. That element denotes an output probabilities array with a specific name, which is the name of a location label assigned to a `casels' instruction or to a `choice' instruction block in an assembler program.
• Added field `are_lines_around_multiline_param' to structure `qsmm_dump_prg_desc_s'. A non-negative value of that field specifies to insert empty lines before and after multiline user and `casels' instructions. The field should be used instead of deprecated field `are_lines_around_multiline_casels'.
• Added field `margin_right_param' to structure `qsmm_dump_instr_desc_s'. A positive value of that field specifies a right margin column for lists of arguments of user and `casels' instructions. A non-positive value indicates not to split the lists into multiple lines based on a right margin column. The field should be used instead of deprecated field `margin_right_casels'.
• Added field `nline_param' to structure `qsmm_dump_instr_desc_s'. The value of that field is set by function `qsmm_instr_str'. The number of lines of arguments list of an instruction printed is written here. The field should be used instead of deprecated field `nline_casels'.
• Changes for assembler programs processing:
• Fixed the bug that could cause the values of output probability variables retrieved by function `qsmm_get_node_var_prob_out', the values of elements of output probabilities arrays retrieved by function `qsmm_get_node_array_prob_out', and probabilities in `case' instructions (in a program disassembled using an assembler program template) to make the sum of probabilities of choice alternatives be greater than 1. Now the renormalized values are retrieved for which the sum of probabilities of choice alternatives is equal to 1.
• When retrieving probabilities of type QSMM_PROB_FQ by functions `qsmm_get_node_var_prob_out' and `qsmm_get_node_array_prob_out' or by disassembling a node using an assembler program template, values proportional to observed frequencies will be fetched, not proportional to frequencies weighted by theirselves, as in the previous package release.
• Function `qsmm_get_node_array_prob_out' now supports retrieving a remaining probability that corresponds to a choice alternative just after a `casels' instruction or a `choice' instruction block via accessing an element of output probabilities array with an index equal to the number of location labels in arguments of the `casels' instruction or to the number of `case' instructions in the `choice' instruction block.
• Added functions `qsmm_get_node_var_prob_mat' and `qsmm_get_node_array_prob_mat' that retrieve the type of a matrix, to which an output probability variable or an output probabilities array of a node corresponds. A type of matrix is defined by an element of enumeration `qsmm_mat_e'. If the node is a user of a source probability profile provided by another node, then there will be retrieved a type of matrix of the output probability variable or the output probabilities array of the latter node.
• Added functions `qsmm_get_node_var_prob_cycle' and `qsmm_get_node_array_prob_cycle', which retrieve aggregate statistics on cycle types that correspond to an output probability variable or an element of an output probabilities array of a node. The aggregate statistics is returned in an instance of structure `qsmm_cycle_s' and in an array of structures `qsmm_cspur_s'. If the node is a user of a source probability profile provided by another node, then information on how to retrieve the aggregate statistics will be taken from the latter node.
• Got rid of the idea of a hint for reusing a Huffman tree by a large actor when the same probabilities list is referenced in another `casels' instruction in an assembler program. Now all lists of probabilities defined either explicitly by `probeq' and `probls' directives or implicitly by `jprob' instructions and `choice' instruction blocks are stored in the hash table and are reused automatically without any hints.
• The list of arguments of a user instruction can now be splitted into multiple lines in an assembler program. To indicate that a line of arguments list is continued on the next line, terminate a line, which is continued, with a comma after an element of arguments list.
• The assembler preprocessor now recognizes `line' directives. For every such directive, it adjusts current line number, adjusts current file name (optionally), and emits a corresponding `line' directive for the assembler.
• Provided support for escape sequences `\xHH' in string literals. When preprocessing or parsing an assembler program, every such escape sequence will be replaced with a character that has hexadecimal code HH.
• The assembler preprocessor now correctly converts expression "#__UNIQUE__" to a string literal.
• Functions `qsmm_preprocess_asm_source_file' and `qsmm_parse_asm_source_file' now stop reading an input file when EOF is reached and do not use file length to determine the number of bytes to read.
• Behavior of functions `qsmm_preprocess_asm_source_*' is now specfied when they return a buffer with a preprocessed program text of size greater than INT_MAX: those functions shall return INT_MAX as the size of the buffer.
• Function `qsmm_reg_var_prob' will now raise QSMM_ERR_NOSYS only if both engines are represented by large actors.
• Functions `qsmm_get_node_var_prob', `qsmm_set_node_var_prob', and `qsmm_node_var_realize' now do not raise QSMM_ERR_NOSYS.
• Internationalization support:
• Functions `qsmm_preprocess_asm_source_buf', `qsmm_preprocess_asm_source_file', and `qsmm_preprocess_asm_source_stream' now correctly preprocess assembler programs supplied in multibyte charsets, e.g. UTF-8. To accomplish this task, they internally convert an assembler program text to a wide string according to category LC_CTYPE of a current locale before the preprocessing, produce a preprocessed program text in the form of a wide string, and finally convert the preprocessed program text to a multibyte string according to the current locale.
• Functions `qsmm_parse_asm_source_buf', `qsmm_parse_asm_source_file', and `qsmm_parse_asm_source_stream' now correctly parse assembler programs supplied in multibyte charsets, e.g. UTF-8. To accomplish this task, they internally convert an assembler program text to a wide string according to category LC_CTYPE of current locale.
• Added new error code QSMM_ERR_ILSEQ, which indicates that an invalid byte sequence was encountered when converting a multibyte string to a wide string or when converting a wide string to a multibyte string according to category LC_CTYPE of current locale.
• Functions `qsmm_preprocess_asm_source_buf', `qsmm_preprocess_asm_source_file', and `qsmm_preprocess_asm_source_stream' will now raise QSMM_ERR_ILSEQ when they fail to convert an assembler program text in the form of a multibyte string to the form of a wide string or vice versa.
• Functions `qsmm_parse_asm_source_buf', `qsmm_parse_asm_source_file', and `qsmm_parse_asm_source_stream' will now raise QSMM_ERR_ILSEQ when they fail to convert an assembler program text in the form of a multibyte string to the form of a wide string.
• Function `qsmm_get_node_state_name' can now raise QSMM_ERR_NOMEM or QSMM_ERR_ILSEQ when it fails to prepare a state name in the form of a multibyte string from a state name in the form of a wide string.
• Function `qsmm_get_node_state_by_name' can now raise QSMM_ERR_NOMEM or QSMM_ERR_ILSEQ when it fails to prepare a state name in the form of a wide string from a state name in the form of a multibyte string.
• Functions `qsmm_get_eh_instr_param_str' and `qsmm_get_instr_class_param_str' can now raise QSMM_ERR_NOMEM or QSMM_ERR_ILSEQ when they fail to prepare a textual representation of instruction parameters in the form of a multibyte string.
• Function `qsmm_set_eh_instr_param_str_f' will now raise QSMM_ERR_ILSEQ when it fails to prepare a textual representation of instruction parameters in the form of a wide string.
• Function `qsmm_get_instr_class_name' can now raise QSMM_ERR_NOMEM or QSMM_ERR_ILSEQ when it fails to prepare an instruction class name in the form of a multibyte string.
• Functions `qsmm_find_instr_class_in_set_f', `qsmm_find_instr_class_in_set_fv', `qsmm_get_instr_class_weight_by_name_f', and `qsmm_set_instr_class_weight_by_name_f' will now raise QSMM_ERR_ILSEQ when they fail to produce a formatted instruction class name in the form of a wide string.
• Function `qsmm_node_call_default' will now raise QSMM_ERR_ILSEQ if it fails to convert a textual representation of instruction parameters to a multibyte string when filling structure `qsmm_except_outcome_s'.
• Changes for sample program `test':
• The program now uses a pair of explicitly created actors instead of actor pair.
• The program now supports finding an optimal cycle using an algorithm analogous to Viterbi one. A new value `c' of argument of option `-C, --ncycle-max' indicates to check the connectivity of a state graph of the automaton and to find an optimal cycle in the graph using that algorithm. Another supported value `cs' of the argument indicates to check the connectivity, to perform the simplification of the automaton, and to find an optimal cycle using that algorithm.
• Argument `3' of option `-P, --relprob-type' is now supported. That argument corresponds to type of relative probability function QSMM_RELPROB_BUILTIN3.
• Removed support for option `-a, --auto-spur'.
• Changes for sample program `asm-disasm':
• Option `-L, --large' now supports specification of the size of the pool of probabilities list in normal form for every engine of the model. The format of argument of the option is "PE|,PO|PE,PO", where PE denotes the size of the pool for the environment state identification engine and PO denotes the size of the pool for the instruction emitting engine. If the size of the pool is not specified, then a small actor will be used for the corresponding engine.
• Added option `--prob-prec-var=INT' that specifies the number of digits after a decimal point to print for values of output probability variables and elements of output probabilities arrays. If an argument of the option is positive, then fixed-point notation will be used. If the argument is negative, then exponential notation will be used, and the number of digits will be equal to the absolute value of the argument. Zero argument specifies the default mode.
• Removed support for option '-2, --nchoice-2', because this mode is now deprecated.
• Changes for other sample programs:
• Added program `labyr2', which is a new example of using the Actor API included in the package manual. The program uses the Curses library to show movements of agent in a labyrinth which picture is hard-coded in the program source text using a subset of ASCII characters that look like pseudographics.
• Program `apsamp' now uses a pair of explicitly created actors instead of a `qsmm_actpair_t' object which API became internal one. Various schemes of use of the pair of actors are supported. A scheme is selected by defining one of macros MODE_XX_XX when building the program.
• Program `asmat' now supports various schemes of use of a pair of engines that correspond to the multinode model. The schemes are analogous to those supported by program `apsamp'. A scheme is specified by defining one of macros MODE_XX_XX when building the program.
• Program `maze' now uses large actors for the environment state identification engine and the instruction emitting engine. To make this
• possible, the program now does not use instruction classes weights to prevent moving to obstacles. When the agent moves to an obstacle, a special outcome is returned by the `move' instruction and path length is not incremented. In a final path printed by the program, moves, which were not performed because of obstacles, are denoted by "x". Adjustment of temperatures of the engines is not supported.
• Program `maze-mt' was rewritten and produces the same results as program `maze'.
• Added program `maze-asm', which is a new example of working with an assembler program. An agent, which behavior is modeled by the sample program, solves the same task as in program `maze'. However, the picture of the labyrinth is hard-coded in the program source text as in program `labyr2'. Movements of the agent at the last visiting the labyrinth are shown using the Curses library. A profile assembler program `prg_maze' can be used to assist the agent in solving the task. At the end of its run, the sample program creates file `prg_disasm' with a learnt disassembled program that controls movements of the agent in the labyrinth.
• Program `labyr' was modified to use two explicitly created actors instead of C function node classes and moved to the `tests' directory. Chapter "C Function Node Classes" was removed from the manual, so this program no longer demonstrates a special feature of the package.
• Removed support for options `--trace-all' and `--trace-actpair' from program `fw-simple', because they would turn on the use of deprecated flags of model execution tracing.
• Programs `optact-p' and `tohuff-test' now use M_SQRT2 instead of 1 as the maximum possible distance between two normalized vectors. This changes values printed in `% ef2' columns of test logs (amongst other values).
• Program `optact-p' now supports argument `3' of option `-P, --relprob-type'. That argument corresponds to type of relative probability function QSMM_RELPROB_BUILTIN3.
• Slightly changed a description of one of input vectors in program `optact'.
• Program `optpath' now uses a relative probability function of type QSMM_RELPROB_BUILTIN3.
• To make the source code of programs `optact', `optpath', `fw-simple', `parse-asm', `tohuff-test', and `optact-p' easier to understand, macro CHK_FAIL is now used where possible for checking for errors.
• When multiple models are used by program `predict-test', a signal that has maximum value of weight[ii]*tmd/fq_in[ii] is selected as a resulting predicted signal. Here weight[ii] is the total number of times signal `ii' was predicted by all models at current step, `tmd' is the number of processed signals from an input sequence, and fq_in[ii] is the number of times signal `ii' did occur in the input sequence. As a rule, such method provides greater increase (compared to program `predict-test' included in the previous release of the package) in the number of correctly predicted signals when more models of the same structure are used by the program.
• Added program `langlearn-test' which learns a symbol sequence produced using a probabilistic context-free grammar (PCFG) read from a file. To evaluate learning efficiency, the number of correctly predicted symbols is compared with an upper bound equal to the number of symbols that can be correctly predicted by any means. To learn a language, the program uses plain actors that operate concurrently. The greater is the number of actors, typically the greater is the number of correctly predicte signals. The algorithm is a slightly simplified version of algorithm of operation of program `predict-test' when it uses a uniform probability profile of type 0.
• Removed program `asmenv'.
• Changes for the `configure' script:
• Removed options `--enable-debugging', `--enable-profiling', and `--enable-coverage'. Their functionality can now be achieved by passing appropriate values of argument CFLAGS=... to the `configure' script.
• Added option `--disable-long-double'. That option disables the use of long double precision when calculating relative probabilities of output signals choice on systems that support such precision. The option can be used to obtain computation results on systems, which support long double precision, consistent with results obtained on systems that do not support it.
• Added option `--without-curses'. That option disables building sample programs `labyr2' and `maze-asm' which use the Curses library. The option can be used when that library is not available.
• Deprecated features:
• The most of Actor Pair API was made the internal one. The API should not be used in your new programs, except for functions 'qsmm_get_actpair_actor_env' and 'qsmm_get_actpair_actor_opt'; those functions can be used to obtain handles of actors upon which the multinode model is based. The internal part of Actor Pair API was moved to file "internal.h", which is installed to a directory for header files set by the `configure' script. Currently, the internal part of the API cannot be removed from public header files, because of the need to preserve backward compatibility. The functionality of actor pair can be implemented in your programs using a multinode model or by direct creating a pair of actors.
• The concept of C function node classes was made obsolete. Their functionality can be provided using assembler programs or directly created actors.
• Structure `qsmm_except_weight_s' is now deprecated, because the corresponding exception could be raised only from within C function node classes which are now obsolete. Corresponding field `weight' of union `qsmm_except_u' is now deprecated for the same reason.
• Field `unconfrm' of union `qsmm_except_u' is now deprecated too, because the corresponding exception could be raised only from within C function node classes which are now obsolete.
• Functions `qsmm_get_nchoice_2', `qsmm_set_nchoice_2', `qsmm_get_node_nchoice_2', and `qsmm_set_node_nchoice_2' are now deprecated, because they control a rarely used feature, which is not very helpful.
• Field `are_lines_around_multiline_casels' of structure `qsmm_dump_prg_desc_s' is now deprecated. Field `are_lines_around_multiline_param' that hands over more general information should be used instead of it.
• Fields `margin_right_casels' and `nline_casels' of structure `qsmm_dump_instr_desc_s' are now deprecated. Fields `margin_right_param' and `nline_param' that hand over more general information should be used instead of them.