Muse Proxy Changelog

What's new in Muse Proxy 5.5 Build 05

Sep 29, 2022

There are small changes in the JavaScript inserted in the rewritten pages in order to comply with
HTML standard. For example, <SCRIPT LANGUAGE="JavaScript"> was replaced with <script type="text/javascript">. In case you are using automatic comparisons tests for detecting rewriting issues please adjust them to comply with these changes.
FOLLOW_REDIRECTS from source profiles is now read correctly, ignoring the case. For this flag if its value wasn't case sensitive " false" it was finally set to true. So now if "False" is used then this will be treated as false.
For SAML, SSO, LTI, AdminRWP relaying Muse Proxy is correctly dealing with HTTP HEAD requests with responses not having a body, even if they contains Content-Length.
The following behavior was fixed in MuseKnowledge Administrator Console / Manage Applications: when importing a source having an ID different from the profile file name, the import action is saving the profile using the IDENTIFIER + .xml instead of using the file name given
by the CONFIGURATION_FILE in the export metadata. The import seems successful but if you want to access the source, the error "The profile configuration file was not found." was yielded.
After a server restart, loading persisted sessions failed if an application ID with active sessions is no longer available (renamed/deleted/critical configuration issues) before/during a server restart. This is now fixed and the sessions for the rest of the applications are correctly restored.
In Muse Proxy Administrator console, Manage Applications section the buttons under Raw Edit did not get proper focus on recent updates of Chrome and Edge. This is now fixed.
Logging to Debug log, Access log and Statistics log is now recovering after disk full events are resolved, even if more scheduled rotation occurs. Up to now the logging recovered only if disk

New in Muse Proxy 5.0 Build 04 (Jun 21, 2018)

New Features:
A new proxy application template, MKPF, with a new look and feel is available starting with this version. MuseProxyFoundation, the old one is still maintained and available with all the features up to date. On a fresh install Anonymous, MuseProxyFoundation and MKPF will all be installed and available, while on an update MKPF will be installed but not linked to in the Applications.xml file to avoid compatibility issues with another application having the same code and to protect installations with Small or Medium licensing which already have 4 o 8 applications. So the administrators which are upgrading MuseKnowledge Proxy directly via the setup and want to discover the new template will have to edit Applications.xml and add this entry under the APPLICATIONS section, taking care to set an integer value to the code attribute.
For the Small Organization installation upgrades we recommend replacing the Foundation or Anonymous entry.
The Administrator Console is now based on a new theme and a new component is available to help the administrator of MuseKnowledge Proxy to manage and maintain the applications and sources. This section can be accessed from the Applications menu via the Manage Applications link as long as SERVLET_ENGINE_ENABLED is set on true. The servlet engine is not active by default because it needs more memory and disk resources and very small installations not needing SAML/SSO or visual administration should still work if memory is an issue. There are visual actions for the common elements and raw view/update actions, where directly the XML files behind are edited in a simple JavaScript editor ensuring XML well-formedness. An application can be edited, copied, exported, imported, backed up, restored, checked or deleted. A source can have its profile edited, backed up and restored, while the other attributes from the Sources.xml file related to a source, such as image, visibility, module, attributes and parameters can be also managed. Source grouping including defining new visual areas and categories is also possible. Authentication to an application can be managed, details for the login module set and their con figuration files can be edited in a raw XML form. The Manage Applications tools work with the last MKPF and old-style MuseProxyFoundation provided with the current setup, while for the existent applications not all the functions will be active.
The new Administrator Console offers a smoother SAML Metadata administration under the same browser tab, and this interface was also updated to the new style.
Starting with this version, MuseKnowledge Proxy supports relaying and filtering the WebSocket Protocol. There are more aspects to consider: 1) the rewrite of the initial address provided to the JS WebSocket constructor (the server end-point) in case this is not relative, 2) setting the filtering and frame serialization mode and 3) the effective filters. It is recommended that filtering is performed only if the frames are containing URLs that need rewriting, otherwise just relay the WebSocket transparently. Also it may not even be the case to do any relay if the process functions with the WebSocket end-point unrewritten.
OLSA support is available for proxy to platform authentication. Open Learning Services Architecture (OLSA) is a comprehensive service oriented architecture initiative that is intended to simplify the effort required to integrate SkillSoft learning services with your Learner Management System(LMS) or portal of choice. A MuseKnowledge Proxy source can be integrated with OLSA API to create the user dynamically and perform the signon. This is based on SOAP requests which are made transparently when the configured source with this integration is accessed by a MuseKnowledge Proxy user. If all the configurations related to the organization are done, with this new feature a user of MuseKnowledge Proxy application can jump into the Skillsoft Skillport 8i platform as an authenticated user of this platform without the need for continuous content proxy rewriting.
In order to reliably support third party libraries for various proxy to platform integrations, a new CLASSPATH structure to create ClassLoaders was added in MuseProxy.xml configuration file. The value of this configuration entry consists of one or more path elements separated by semicolon Each path element may refer to a jar file (if it does not end in slash( /) and it is not a directory), a directory of classes (if it ends in slash(/)) or a directory of jar files (if it does not end in slash(/) but it is a directory).
If an application was configured with the LDAP Authentication module ( ProxyLogin ModuleLDAP) then from the HTML form use the same logon parameter names as in case of ProxyLoginModuleUserPassword. The parameters names to be used can be also mentioned in the LDAP configuration file using the new USED_PARAMETERS with positional PARAM elements - the first one for the user name, the second for the password.
The LDAP Authentication module ( ProxyLoginModuleLDAP) can now be configured with a script to take final decisions based on attributes and membership. Methods isMember(String dn) and isAttribute(String name, String valueToMatch[, boolean isRegex]) can be invoked and authentication can be refused or a certain sources group can be selected. The adjacent setWithUser(boolean flag), setMemberUser(boolean flag), setAnyAttributeValue(boolean flag) and setMemberAttribute(String attrName) methods controls how the isMember(..) and isAttribute(..) operates. More details can be found in the comments from ProxyLoginModuleLDAP.xml and in the manuals.
In case there are more LDAP servers to be confronted for a single application authentication then, for such a scenario, configuring multiple USER_GROUP entries in the ProxyLogin ModuleLDAP.xml (one for each server) and linking to these setting LEVEL on sufficient from the AuthenticationGroups.xml file is possible.
Referral configuration for LDAP authentication is possible via the new REFERRAL element with either ignore, follow or stop values. By default the referrals are ignored. However if there is an instance of LDAP server using referral then set the flag to follow. Note that the same credentials are applied and the host name used in the referral URI must be resolvable. All the normal entries are processed first, before following the continuation references. However a "referral" error response is processed immediately. When this setting is on stop, then when a continuation reference or "referral" error response happens the process stops. Even with the referral setting, in case of Active Directory the Global Catalog (port should be used instead of individual instances (port 389/636). This is because the referrals themselves often contain LDAP server hostnames which differ from the original Active Directory LDAP server (domain controller) hostname. These hostnames must be resolvable via DNS in order for successful resolution, but sometimes they are not. The referral service port must also be reachable via the network through firewalls, etc.
New attribute flags for TLS related settings are available for the LDAP_URL in case of LDAP authentication. These are startTls="false/true" and sslTrustAll="false/true", ensuring a wider integration of Muse Knowledge Proxy with LDAP servers.
Proxy Host and Port can be set during SAML, LDAP and SSO login depending on the authentication properties, for example based on a group property coming from SAML we can choose a certain source IP for outbound. The sources will have to be configured using PROXY_USED set on LOGIN_LEVEL. The Login Level script section will have to set proxyHost and proxyPort accordingly. For outbound IP which is assigmed to the proxy machine itself the port must not be specified.
Starting with this version the SAML Configuration can be refreshed without restarting the entire Servlet Engine. In the Administrator Console under Configuration menu - SAML Authentication page identify the Refresh SAML Configuration button and click it to refresh the SAML Configuration after performing an edit to the securityContextmetadata.xml, adding a new keypair or a new SP/IDP metadata. Reloading the SAML related beans and context required a different organization of some beans in order to be reloaded and to avoid memory leaks and that is why in case of an upgrade the instructions displayed at the end of the setup (or in ${MUSE_HOME}/Upgrades.txt) must be followed. Most of them are related to the file
The SSO Configuration can be also refreshed without restarting the entire Servlet Engine. In the Administrator Console under Configuration menu - SSO Authentication page identify the Refresh SSO Configuration button and click it to refresh the SSO Configuration. As in the case of SAML, for an upgrade there are certain modifications imposed related to the Spring configuration elements. For example applicationContext.xml and securityContext.xml must be moved one level up. Also check that in MUSE_HOME}/proxy/webcontexts/ssoRWP2/WEB-INF/web.xml the param-value cor responding to contextConfigLocation is now /WEB-INF/applicationContext.xml,/WEB-INF/securityContext.xml.
The Apache JCS component used for the mixed memory-disk navigation system storage (when NAVIGATION_SESSION_STORAGE has the value JCS) was upgraded to version 2.2.
In addition to the ECMA script processing via the Oracle Nashorn engine, the reference process in a source profile can now be done via BeanShell scripting engine. This is lighter with regard to the metaspace memory consumption because it is not using LambdaForms but classic reflection calls. This can be configured in a source profile by providing beanshell as the value for the new type attribute of the DEF configuration entry, for example DEF src="MyScript.bsh" type="beanshell"/>
From within the ECMA(JS)/Beanshell script used within a source profile for processing a console object is available to log messages in the MuseProxy.log file on various levels: console.log() writes on NOTICE, console.warn writes on WARNING , console.error writes on ERROR and console.debug writes on DEBUG level.
Besides ECMA and Beanshell, Java classes located in the CLASSPATH configured for the source can be invoked by using the " java:" prefix in the value of process attribute
The integration within a Learning Management System (LMS) was extended and starting with this version, a MuseKnowledge Proxy application can be integrated as Rich Content Editor in a Canvas LMS. Note that only the latest MuseProxyFoundation and MKPF applications can be integrated. Detailed guidelines for this integration are available in Configuration menu - SSO Authentication page from Administrator Console by accessing the LTI Guidelines. is a new pattern option with the scope to exclude from main source(Link Out) but rewrite with others if matching. This option can be used within include and exclude pattern options from REWRITTING_PATTERNS configuration entry from the source profile. excludeLocal. When accessing an expired application a different HTTP status can be presented instead of Not Found. This is configured using the new element EXPIRY_ERROR inside the Ap plications.xml file <EXPIRY_ERROR status="410">true</EXPIRY_ERROR>. For an upgrade check if this element was automatically added in the existent Applications.xml file by the setup.
If the default HttpModule source module implementation is used, in case an empty body HTTP error status is received from the server with the first request, it is now forwarded to the browser this was previously done only when a body was present, too.
A new attribute method was added for URL configuration entry from the source profile to specify which HTTP method to be used for the request done - each URL part of an extract and navigate scenario can have its own method attribute (if empty GET or POST is assumed). One of the following values can be used: GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE or PATCH. The following configuration entry represents an example of usage for this attribute: <URL method="PATCH">http://httpbin.org/patch</URL>
The HEADER element is a new configuration item in the source profile aiming to set HTTP headers for each preliminary request. This option may appear several times under one URL configuration entry. This element has two attributes: name which represents the name of the header and value representing its value which will be set and this can contain variables captured by an extraction group or configured in the profile. Because the URL configuration entry may occur several times, only the HEADER elements located under the first URL entry will be send at the first request and the HEADER configurations located under the next URL will be send at next request.
An option to refresh java policy files was added in Administrator Console and can be found under Advanced menu - Operations page. Java Policy Files which will be refreshed by this action are ${MUSE_HOME}/java.policy and ${MUSE_HOME}/jaas.policy. An error message will be displayed in case a file is malformed pointing to the erroneous line.
The backend Jetty and all of the web applications running inside are configured with an extra layer of security.
Because some browsers, such Microsoft Edge, don't send the cookies in a request for the favicon image, the response is a redirect to the login page and because the login process is initiated it adds unnecessary pressure on the proxy in case of SAML/SSO authentication. To avoid this favicons for Type 3 are transformed in absolute links and left unrewritten.
Error templates in all the contexts are now neutral, without any branding and without any external resources to CSS, JS or images. They resemble the simple browser style used to report errors.
For source profiling a new boolean flag, DYN_HOST_MAPPING is available. It indicates that host mapping to a shorter generated value will be used on the fly if the https version of the host label exceeds 63 characters (even if the host is requested on plain http:// as later it may change on https://). Similar to HOST_MAPPING, for this option to take effect it is required to use Utilities / Evaluate Shortcut URL / Update Mappings or wait at most the MuseProxy.xml's REFRESH_INTERVAL period (default 5 minutes). HOST_MAPPING have priority over DYN_HOST_MAPPING when it is decided if a host will be mapped.
A grace time for the links encoded via rwpState during the SAML/SSO authentication was added in ${MUSE_HOME}/proxy/webcontexts/Services/profiles/TinyURL.xml. After being consumed the links are not expired immediately after their first usage, but rather after the specified number of milliseconds within the element RWP_STATE_LINGER. This ensures that the URL with rwpState in it, that was saved before the login flow starts and which encodes the initial request to Muse Proxy, can now survive more after being consumed for the first time. In turn, this ensures that, if somehow the back button is pressed on some browsers, or the Load Balancer times out and the user press enter/refresh on the URL in the bar, that link will still work.
Bug Fixes:
The Cookie header could be duplicated in the next URL requests from the extract and navigate scenario in case of using HTTPModuleApache as a source module. This is now fixed.
While performing the initial source request, an invalid Set-Cookie header field (for example no pair just Set-Cookie: HttpOnly;Secure) is no longer triggering the failure of the entire source.
Restarting the Servlet Engine used for SSO and SAML can lead to Thread and Classloader leak due to the usage of java.util.Timer for each metadata. A configuration change to use a single Timer, which is canceled when the server is destroyed, resolves this. For an upgrade, the file securityContext-metadata.xml must be updated to replace
Improved the relaying of binary responses represented as chunked transfer encoding streaming.
Correctly escape the cookies inserted as the content of _rwpSessionCookies - also backslashes and slashes, besides quote and apostrophe.
Fixed a concurrency bug when deleting heavy usage.
Fixed cache issues when an outbound source IP is used for a source and when no is specified.
HTTP requests with HEAD method containing the Content-Length header are now correctly working.
The REMOVE_COOKIES configuration from MuseProxy.xml worked only if there was one cookie; starting with this version this is fixed and multiple cookies are now correctly removed. This item is needed because some load balancers adds security cookies but never deletes them.
Changed the algorithm which parses html pages to skip @import rule from div elements because it is no more used by the CSS Style Attributes standard and usage of CPU could have increased on big pages.
DEF src Connections and scripting attribute values were not read if DEF's content was blank - this is fixed.

New in Muse Proxy 4.4 Build 02 (Jun 2, 2017)

New features:
Muse Proxy can now create access log files in the same configurable format as those created by standard web servers such as Apache HTTP Server, format which can be set via a style pattern an extension of the Common Logging format). In order to do this, the LOG_FORMAT element should have the type="apache" attribute set. To have a good base for statistical information, especially in a multi-tenant environment, we recommend using more items besides Common Logging, by adding the inbound server IP address, Muse Proxy application, user session, content type: LOG_FORMAT type="apache">%h %A %w %W %u %S %t "%r" "%{Content-Type}o" s %b</LOG_FORMAT>. On a fresh installation this format is already set up, while on an upgrade the old format is left in place to keep compatibility in case there are external log parsers set. If a different output is needed then more information can be found in the document MUSE_HOME}/proxy/doc/Muse Proxy.pdf in section "7.2 The access Log".
Introduced support in ProxyLoginModuleSQL.xml for SQL statements. More details on how to use this feature are present in Muse Proxy Advanced Configuration.pdf section 6.4.5.6 ProxyLoginModuleSQL. The backward compatibility for specifying a table is kept.
Added support for IP ranges in ALLOW and DENY rules from ProxyLoginModuleIP.xml, and this range will be matched against the IP address the connection is coming from. Both IPv4 and IPv6 are supported. All types of rules can be mixed if need be, for example one allow/deny rule can be a wildcard such as 217.156.14.* another rule can be a CIDR rule such as
217.156.0.0/16 and another one can be expressed using the range 217.156.11.0-217.156.15.255
Introduced redirection to remote Sources depending on the end-user IP (non-proxied links). This is done via Sources.xml file via the new
REDIRECT section containing IP_RULES elements which are applied on a set of sources and, if the request is for a source that matches the APPLY pattern and the request's end-user IP satisfies the ALLOW DENY sequences, then the response will be a native redirect to the source URL.
Source parameters can be provided via Sources.xml level in the <SOURCE> element via multiple <PARAMETER name=""> children. Each parameter can be further referred in the Source Profile in <URL> or <POST_PARAMETERS> via ${name} syntax and its value will be resolved to the content of this node, exactly as if it were defined inside the source profile.
The Single Sign-on Authentication (other than SAML) core was upgraded and part of the new features Central Authentication Service (CAS) is also supported. Upgrades instructions related to how to handle the changes in securityContext.xml are provided via the setup.
Keep up with SHA1 deprecation.
To avoid DNS limitation of 63 bytes per label in case of proxing https://hosts via Rewrite by Host technique using https://proxy URLs, there is now the possibility to map the very long FQDN to shorter names by using HOST_MAPPING elements in the source profile.
This release contains an improvement for Multi tenant environments using the same host name for all tenants but individual IPs for each one. A chaining request to proxy itself on the different IP is no longer taking place for each request, rather the outbound IP is set directly to the one of the chaining proxy. To achieve this only the PROXY_HOST must be configured to the allocated IP in the source profile or at the application level and the PROXY_PORT must remain empty.
The generation of the Client Session cookie values was improved.
The generation of the Connection ID values was also improved.
Implement an improvement when action=source is called, and data is read from the Sources.xml application file.
Refine the errors "Unexpected exception while accessing target source. " to contain more details about what was wrong with the access.
Added a limit for the size of multipart POST requests which will be kept in memory and requests which will exceed this limit will be temporary saved on the file system. The value of the limit is configured in USE_MULTIPART_TMP_AFTER tag from ${MUSE_HOME}/proxy/MuseProxy.xml.
Added COOKIE_PASS_PATTERNS options and cookies with name matching these patterns are passed into the browser even if they have a domain. This configuration must be used with care and only where strictly required. Cookie name patterns can be specified such as SESSION*, separated by semicolon (;).
Secured the /admin context availability.
Related to proxy application IP Authentication, the REVERSE flag was added to control if reverse DNS is performed for the end-user IP. The trend is to set it false in the configuration files from now on. Reverse DNS is too costly and can slow down the authentication process.
In case of HMAC, Referer and IP login modules there's no ID to check against a database and hence nothing to be written in the log file, however in case of various integrations we could be receiving a special parameter in the request for tracking purposes and we want to keep this for logging. Now these login modules also support the LOG_USER_ID configuration entry, for
example: <LOG_USER_ID>${token}</LOG_USER_ID>
Because JSESSIONID name is too general, the session cookie names for the embedded Jetty contexts (related to Single sign-on) were changed.
Added encodeURIComponent and decodeURIComponent to be used for reference and parameter process for first source requests (including extract and navigate scenarios). The functions are compatible with the JavaScript ones. The existent encodeURL and decodeURL are based on JDK URLEncoder/URLDecoder which are using application/x-www-form-urlencoded MIME format which is not entirely the same as the URI encoding which, for example transforms space into %20= instead of +, for example and some servers are sensitive to these differences.
Bug fixes:
Microsoft Azure AD OpenID Connect End Point v2.0 and End Point v1.0 can now be used for authentication without complex workarounds - guidelines and suggestions are available in the Ad ministrator Console in Configuration / SSO Authentication.
Security constraints for SAML or SSO authentication when starting from plain http://proxy links are by default not enforced.
HttpModuleApache is now correctly sending post data for the extract and navigate scenarios as URL encoded data.
Filtering on the exact Client Session ID in the Administrator Console was corrected.
Fixed parameter name encoding for HMAC Link Generator page inside Administrator Console.
Avoid redirect loops when a Type 2 (rewrite by path) link expires for SSO2 (OAuth / OpenID) authentication.
Correctly persisting Tiny URLs, that are used in some cases for MuseKnowledge Search integration.
Fixed memory usage when downloading more log files from the Administrator Console.
Quicker release of file descriptors when local resources such as images, javascript or css are served by Muse Proxy.
While rewriting object embed and param elements the protocol relative URLs (the ones starting with are correctly treated.
The starting point Type1 links used for MuseKnowledge Search integration can also be generated on https://protocol.
Existing navigation session were ignoring the FIND/REPLACE filters after a mnm.jar update - this is now fixed.
A very rare deadlock appearing when a Muse Proxy under extremely heavy usage has its mnm.jar updated. This fix is actually carried by mnm.jar version 1.513 itself, so old versions of Muse Proxy can be updated without a full upgrade to fix this.
Interpreting the request Forwarded headers was failing if no proto= was found in any of these headers. This is now fixed.
The following rare case was fixed: if the same linkout source is in two different applications and the same end-user accesses both applications from the same browser at the same time it is possible that the source we link to is used via a navigation session from the other application yielding misleading statistics.
Protocol relative URLs in HTTP redirects could have generated wrongly rewritten links - this was fixed.
In case no ENCODING is defined in the source profile, first source request(s) (such as extract and navigate) are now considering UTF-8 as a default encoding for various processing such as parameter encoding processing or deserialization from gzipped content (also when no charset is present in the Content-Type reply).
All multiple HTTP response header fields are now reaching the browser in the response to the first rewritten URL (by host or by path) request which is initiated by the source request action=source url=

New in Muse Proxy 4.3 Build 02 (Dec 9, 2016)

In the family of Single sign-on authentication, besides SAML2.0, MuseKnowledge Proxy now supports a wide range of OAuth, OAuth2, OpenID Connect SSO based. MuseKnowledge Proxy supports connectivity with more than a dozen of OAuth providers and also a generic OAuth client implementation can be configured for authentication to the providers that are not diverging from the usual practices in OAuth requests and responses.
External HTTP Authentication Login Module for MuseKnowledge Proxy is available.
Introduced experimental support for load balancers that do not spoof/masquerade the IP of the end-user and pass it in the protocol layer via HAProxy PROXY Protocol v1 or via X-Forawarded-For.
Category grouping for source layer presentation is now possible. Multiple areas can be defined, including A-Z ones and these are displayed in different tabs. Integration with MuseSearch passthrough is available if dblist source attributes are defined.
Introduced experimental support for follow-up links without authentication (session cookie) such as for the preflight OPTIONS where the standard requires the browser to avoid sending au thorization data. Only links that are generated by a valid navigation session are allowed and only if the HTTP method and link matches the new source configuration element value.
Apache HTTP Client library can now be used for the first source request (extract and navigate scenario). Because the Oracle JDK URLConnection does not allow the control of the outbound IP address up to now we were forced to perform an extra request through MuseKnowledge Proxy and this increase the complexity of troubleshouting and authentication configuration and adds an extra request. The Apache HTTP Client allows control over the outbound IP address and there's no need of an extra request.
dded a limit which usually triggers using temporary files for saving streams of bytes in certain cases, for example for performing gzip. Otherwise these operations are performed in-memory, and, although more time-efficient, this can limit the number of concurrent requests. This controlled via the new flag USE_TMP_FILE_STREAM_AFTER in ${MUSE_HOME}/proxy/webcontexts/NavigationManager/profiles/NavigationSession.xml
Added a new tool in the MuseKnowledge Proxy Administrator Console - HMAC Link Generator - for generating HMAC links for testing the login via HMAC (keyed-hash message
authentication code) signing. The utility allows specifying all possible parameters and combinations for generating a HMAC link.
Added a new tool in the MuseKnowledge Proxy Administrator Console - Evaluate Regex - for evaluating regular expressions. The utility is most usefull for administrators to troubleshoot sources filter configurations for find and replace. It has two forms: By JDK RegEx and By Running Filter . The By Running Filter tool generates the XML snippet that can inserted into the MuseKnowledge Proxy source profile.
For load balanced environments the ID value (if defined in MuseProxy.xml and if cookieSuffix="true" - which is true by default, if missing) is also added to the session cookie
name as a suffix (e.g. MuseProxySessionIDp2), because this behaves more reliable in certain Load Balancer cases, such that the ones combining routing rules (/admin rule or p1.[a-z].http rule to go to a certain proxy) with the sticky cookies mechanism.
Log files can be named containing elements of the creation date, based on the new pattern attribute of the LOG element. For example to capture the activity on a daily basis in files such as access- 20161123.log and keeping a maximum of 365 such files the corresponding logger will have to be configured as below in MuseProxy.xml.
Bug fixes:
The PUT HTTP method was relayed as POST and some AJAX implementations may not accept this. This was corrected.
A fix related to file uploads through rewritten POSTs was considered.
Digest authentication for remote sources was not working correctly if qop (quality of protection) value is not wrapped in quotes. Although this behaviour was according to the RFC, MuseKnowledge Proxy is now more permissive in this regard.
IP rules for the application level ProxyLoginModuleIP login module are treated the same way the rules for ${MUSE_HOME}/proxy/hosts.xml are, that is the first rule that match counts.

New in Muse Proxy 4.2 Build 02 (Jun 22, 2016)

New in Muse Proxy 4.1 Build 01 (Jul 2, 2015)

New features:
Introducing shortcuts to Muse Proxy Source Entry Point URLs - Muse Proxy offers now new Entry Points for source navigation without the need to specify the sourceID and action=source parameters, rather just an url parameter (or its encoded form). Both approaches have their advantages, the original one being more secure as it can hide the initial URL. However for integration purposes the direct url parameter is more suitable. Note that based on the url value provided the corresponding sourceID is selected. This is done based on a search mechanism against the defined sources described in the documentation, hence Muse Proxy will not automatically proxy any URL provided in the url=parameter. Not that it wouldn't be able, but it would be dangerous.
Availability of a new Muse Proxy Admin function Utilities / Evaluate Shortcut URL to test what sourceID is discovered for a certain application when a certain url would be provided as parameter. This is needed as the order and definition of sources matters in choosing the source definition that will be further used.
Sources can be hidden in the Muse Proxy application interface but usable in Entry Point URLs (either with explicit sourceID or implicit shortcut URLs).
A new REDIRECT source configuration element is available and its primary purpose is to be used in a hidden "fallback" source to identify the pattern domains of web sites with free content that do not have to be proxied, and that can still end-up in Shortcut Entry Point URLs generated by external systems. Another usage may, in the future, be for sources that integrates without the need of rewriting which, after a first request, are redirecting with an authentication token to their domain and that must not be rewritten at all
Extract and Navigate options are available for a source profile in order to deal with sources requiring more authentication or navigation steps (such as selecting a certain database having a dynamic URL containing session identifiers) before handing the control to the browser. Extract variables via the new EXTRACTOR option from content and using them in the next URLs or next Posts are the key to this scenario.
A new boolean source option, SHOW_GET_PARAMETERS, is available in case GET parameters of the source URL need to be revealed when the control is handed over via the follow-up URLs. By default they are still hidden for security reasons.
In order to troubleshoot, or avoid extra configurations, a new boolean source option, SSL_TRUST_ALL, can be used for HTTPS sources which involve certificates signed by CAs not covered by the main proxy trust store which is now a copy of the JDK 1.8.0_45. Up to now for these cases a manual configuration of certificates is required and this may not be straight forward. However, note that trusting source certificates that are not signed by trusted CAs is a security decision.
Application context mapping was extended to support more contexts for the same application and can now use the host and port in the CONTEXT_ACTIVATION_RULES/URL_RULES/URL_PATTERN field. This means that an application can be configured to respond to multiple paths and for hosted environments the DNS name of each organization can be used to distinguish between applications in case each or ganization has a single application and the same path (e.g. /rewrite). By default MuseProxy Foundation comes configured with /MuseProxyFoundation and /rewrite patterns.
Introduced support for persisting the client sessions and their corresponding navigation sessions, authentication tokens and tiny URLs during a graceful Muse Proxy restart. Persistence is controlled through a new boolean flag, PERSISTENCE, in MuseProxy.xml main configuration file. Session and authorization data is persisted as long as the shutdown happens gracefully, that is via the Muse Proxy stopping scripts, Windows Service stop, or via ^C or SIGTERM but not through a forced process kill or unexpected machine crash.
Configurable Find and Replace filters acting on the HTTP body can now be crafted in the XML source profiles and will be interpreted at run-time, without the need to write Java code. There are two types of filters: regular expression based and Muse Proxy token rule based similar to the token rules written in Muse Proxy Java filters. There are simple (just find/replace) and complex filter configurations involving conditions (such as APPLY_IF_FIRST) and variables.
Introducing an alternative Navigation Session storage which uses disk space to spool idle sessions. As Navigation Sessions are in the center of the navigation process and each new entry point URL translates into a Navigation Session in order to make room for more navigation actions without adding substantial RAM Muse Proxy can be configured with Apache JCS - Java Caching System system for a composite LRU cache (memory/indexed disk). By default navigation sessions that weren't used for 5 minutes are spooled to disk (but not expired) and if they are requested until they time out then they are retrieved back in memory. For the indexed disk storage only the keys will be stored in memories, but not the content.
The usage of the JCS system is possible through a new option NAVIGATION_SESSION_STORAGE available for setting in ${MUSE_HOME}/proxy/MuseProxy.xml file. This new option must be set to JCS so that the hybrid storage is used. Otherwise, the default value is memory.
The JCS setting is recommended for hosted environments with dozens of institutions in case RAM memory proves to be a limit.
Application Web Contexts are also visible in Monitoring/Client Sessions section from Muse Proxy Administrator Console.
Extend the Admin Utilities / Encrypt Password to support the symmetric DES encryption.
Icon configuration for each source is now available in any MuseProxyFoundation based application. If configured, the image will be displayed under the Source name, next to the source description.
The Client Session ID encoding was changed from Hex String representation into base 36 representation in order to lower it and inherently the space used.
Cookies set from the JavaScript level which aren't intercepted by Muse Proxy JavaScript _rwp wrappers were saved in the navigation session. This is not generally necessary as those cookies are usually used at JS level only and besides this could save memory for storing more navigation sessions in parallel. Flags for controlling the un-intercepted JS cookie polices were added in MuseProxy.xml and each source could actually overwrite these in its profile. The new options are COOKIE_JS and COOKIE_JS_PERSIST.
Some of the Navigation Session attributes such as Original URL, Entry Point URL and the associated cookies are stored as byte arrays instead of Strings thus reducing the memory space for a Navigation Session.
Some sources report the JSON content as text/json instead of application/json and now Muse Proxy recognizes this, too, although the standard value is application/json.
Bug fixes:
Corrected the memory values listed for Client Session sizes as the size accounted for shared configuration data, too.
The tilde (~) character was encoded when rewriting location redirects when it should not be as it is not a special character. However tilde (~) is encoded because JDK's URLEncoder considers that browsers (although the old ones such as Netscape) do encode them despite the protocol requirement, the explanation being that “It appears that both Netscape and Internet Explorer escape all special characters from this list with the exception of "-", "_", ".", "*". While it is not clear why they are escaping the other characters, perhaps it is safest to assume that there might be contexts in which the others are unsafe if not escaped”. Muse Proxy now fixes this as modern browsers are no longer encoding the tilde (~) character.
The LDAP login module can now accept a complex query combining more attributes - (&(objectClass=person) sAMAccountName=${NAME}))
Location URLs that did not follow the standard and contain un-encoded slashes (/) in the query part (i.e. after ?) were not successfully rewritten. Although non-standard, Muse Proxy can now cope with them without dropping the query part up to slash (/)
The Rewrite by Host mechanism was expecting a port number after colons (:) even if an anchor such as http://host:/path is equivalent with http://host/path and then navigation of such rewritten URLs was failing as the host part was rewritten to contain ".p" without a value.This is now corrected.
The inbound source address was not used for HTTPS remote connections in case no proxy is configured either at source, application or global level. The default IP address of the machine was used. This is now fixed.
In Muse Proxy Admin, the Monitoring / Client Session / Navigation Sessions the label "Rewritten URL" was renamed into "Entry Point URL" and its value is now correctly computed for the cases of source navigation (up to now it was working only for links coming from MuseSearch).
Fixed a concurrency bug for FreeMarker interface template file loading which could have resulted in sporadic "Resources not found" errors.
Fixed a rare shutdown refusal issue when stopMuseProxy script is used.

New in Muse Proxy 4.0 Build 02 (Dec 23, 2014)

New in Muse Proxy 4.0 Build 01 (Dec 22, 2014)

NEW FEATURES:
Added logic and a new configuration element for skipping content rewriting. There are cases in which resources have to be requested via the proxy (so their URL has to be rewritten), but their content must not be parsed and modified, being served untouched. For the proxy sources this con figuration element is called TRANSPARENT_CONTENT_PATTERNS while for the Muse Search starting point URLs there are two new rules (includeT: and excludeT: to be defined in the NAVIGATION_MANAGER_MODE of the Muse Source profile (editable under Link URLs in the MCAA).
Added support for Search Widgets and Form Integration via Muse Proxy. For this, an extended Muse Proxy source type URL is used at the application level appending an URL parameter (either encoded or direct, non-standard) and an optional parameter stating if the request parameters are further submitted using GET or POST methods. Something as the next URL can be used to replace the initial HTML form action where the value of the
url parameter was usually the initial value of the action attribute:
http://proxy.edulib.com:9797/MuseProxyFoundation?groupID=1&action=sour
ce&sourceID=SourceID&nativeParams=POST&url=http://provider.domain.com/
path/etc
Also, a similar link up to, or including url= can be input to many Provider's Widget Builders. Besides having the default option of sending all the POST parameters to the native source, the administrator can have a finer control.
Prefix the parameters for Muse Proxy with _lrwp_ and these will be used locally for Muse Proxy. All the rest of the parameters will be sent to the source. It is assumed that one knows which parameters to prefix for the Muse Proxy.
Prefix the parameters for the native source with _rwp_. If at least one is prefixed then it is assumed that only the prefixed ones will be sent to the native source - hence all the rest of the parameters which are not prefix will be sent to the Muse Proxy.
If there are both lrwp_ and rwp_ prefixes then parameters with lrwp_ will be locally used for Muse Proxy, parameters with rwp_ will be used for native sources and parameters which are not prefixed will be used locally for Muse Proxy.
Rewrite by Host - Introducing the Rewrite by Host (Proxy by Host) functionality which solves more easily situations where the initial Rewrite by Path mechanism, which stored the proxy markers in the path, altering the path, was colliding with the source scripts assumptions on the URL's path. Although the Rewrite by Path mechanism tried to re-decode the path when needed by the native JavaScript there are cases when it is very hard to achieve this even through a special filter.
Leaving the path untouched and altering the host section of the URL brings in some cases more advantages, while in other cases there are also disadvantages. That is why this option will be configured on a source by source basis, because most of the sources work on the initial Rewrite by Path and so have the advantages of no wildcard DNS changes and no wildcard SSL certificates.
The format of the rewritten links that are navigated in the browser, after the starting point are part of the Muse Proxy technical mechanism and not an API being subject to future changes and should not be used for interconnection purposes or entry points. Currently, besides the native host, the navigation session ID, the Muse Proxy ID and the native protocol are stored as well in the host sub-domain. To cope with DNS fully qualified domain name and token restrictions it is advisable that in case of load balancing the ID configured in MUSE_HOME}/proxy/MuseProxy.xml must be as short as possible (even one letter), and must not contain dots (' nor dashes ('
Because there is no other technical solution for accessing sub-domains, the DNS server from the network where Muse Proxy is installed must be configured, besides the normal host name entry, with an extra wildcard DNS entry so that all the sub-domains of the proxy FQDN point to the same IP, the IP of Muse Proxy. For a hosting scenario more such entry pairs are necessary. For a better performance of the Rewrite by Host and also associated with a hosted solution the main configuration file MUSE_HOME}/prox /MuseProxy.xml requires to list the proxy fully
qualified domain names under the SERVER_NAMES element.
Added support for Load Balancing HTTPS traffic via SSL termination, so that the load balancer takes care of the SSL traffic, while the connection between the load balancer and the Muse Proxies is done in plain text, assuming a secure network, thus avoiding unnecessary encryption times. This is achieved either by the de-facto X-Forwarded-Proto header field or by the RFC 7239's Forwarded header field containing " proto=https" which make Muse Proxy behave as if the inbound connection was on SSL.
Added configuration elements to control the logic of deciding the resulting URL protocol, either HTTP or HTTPS. Up to now Muse Proxy could only isolate the protocols of the source from the protocol of the access. However, accessing Muse Proxy via HTTPS when a source is on HTTP is tricky as all the resources must be accessed on HTTPS, or otherwise the browser security will forbid getting the non-secure resources. This make source configuration or the build of some source filters quite hard. The control of the resulting protocol is done through two new options IF_HTTP and IF_HTTPS that take the value "proxy " or "source". If missing, the "proxy" behaviour is assumed. The options are available both at the global level and at the source level.
If Muse Proxy rewrites an " http://" URL, the IF_HTTP option gives the resultant rewriting protocol based on the URL protocol and on the entry point of the current navigation session. If the value is "source" then the protocol will always be " http", irrespective of the proxy's one. If the value of this option is "proxy" then the protocol is either "http" or "https" depending on the entry point of the navigation session.
If Muse Proxy rewrites an " https://" URL, the IF_HTTPS option gives the resultant rewriting protocol based on the URL protocol and on the entry point of the current navigation session. If the value is "source" then the protocol will always be "https", ir respective of the proxy's one. If the value of this option is "proxy" then the protocol is either "http" or "https" depending on the entry point of the navigation session.
For hosting more organizations on the same Muse Proxy, without SSL Termination on a load balancer, the case in which each organization needs its particular certificate can now be achieved. Individual KeyPairs (Private Key and Certificate) can be assigned independently for each IP. Because the SSL handshake takes part at the TCP/IP level distinct certificates require distinct IPs for the association to take place; distinct ports wouldn't be enough as certificates has to be assigned as well with host names.
This is possible via the extension of the configuration element SSL_KEYSTORE_FILE in the sense that it now allows a new attribute ip="" and the entries with the ip attribute can be multiple. If a SSL connection reaches an IP and that IP does not have an associated SSL Java KeyStore then the default KeyStore file (the one with no ip attribute) is used.
SSL Protocols (algorithms) that are used both on the server end and on the client end (requests against the sources) are configurable. The default configuration is for example not including SSLv3 to avoid the recent vulnerabilities. As Muse Proxy runs inside an Oracle Java Virtual Machine the permitted values for the SSL context and enabled protocols must be in accordance with it. Currently the set "TLSv1; TLSv1.1; TLSv1.2" is configured and depending on the JVM version the available ones are used. For example in JDK 1.6 only TLSv1 is supported. Although not recommended there are some sources that explicitly require the configuration of SSLv3 Care must be taken when configuring these. That is why the SSL Protocols on the client end can be configured both at the system level and at the source level to ensure that the such sources are isolated.
Implemented support to serve the response content encoded using gzip for the Navigation Manager rewritten pages. This is done only if a matching Accept-Encoding is in the request and contains the gzip token. This policy of using Content-Encoding is configurable at a global level.
Enhanced the Connection Refused error messages - Explicitly added in the log file the URLs generating the errors that are obtained when navigating starting from Type1 request (MuseSearch generated full text entry point URLs). Also for the proxy source links some of the headers for the first requests are displayed (this contains the URL and the next proxy hop). Although Java stack traces may print the host name in some cases, thus duplicating some information, always having the explicit URL should make troubleshooting easier.
Instead of redirecting to the native site, the expired navigation URLs that are initiated starting from a proxy source URL can now require re-logon in the initial application and authentication group and, after successfully logging in, the source navigation is restored as long as the native source URL is able to function out of the initial session and request context. This is a global configuration option, and also requires that each Muse Proxy Application and each Muse Proxy Source have stable and unique code s defined in the Muse Proxy application files ${MUSE_HOME}/proxy/webcontexts/Applications/Applications.xml and ${WEB_CONTEXT_HOME}/profiles/Sources.xml. This is required as Muse Proxy is a multi- application server.
The Muse Proxy Client session Cookie has been modified to MuseSessionID and is now set using a domain not a host. This works consistently both for Rewrite by Host and Rewrite by Path.
Some providers require sending the end-user IP via X-Forwarded-For de facto standard field. This is now possible on a source by source configuration, by default the end-user (client) IP not being sent.
Reload and use main configuration elements from the MUSE_HOME}/proxy/MuseProxy.xml file, via a new admin operation Refresh Configuration located under the menu Advanced/Operations/ It is not possible to reload all the elements because some of the objects are only created upon Server start-up, but many of them are reloadable without interfering with the live session.
BUG FIXES:
Proxying HTTPS URLs as a classic proxy ignored the port and always used the default one, i.e. 443. This was fixed as being important for the Muse Search scenarios.
Improved the analyzer for finding the end of script tag to be in full accordance with the HTML Specs detection according to the consortia state machine. The end of script detection is a tricky operation, but it doesn't have to do with JavaScript quotes and comments; normally the end of script is the first /script* any char) but inside a HTML comment one can start another script and /script will no longer be the end of the main script. However, if there's only a /script inside the comment, or before a /script end the first occurrence is a /script not a script then it represents the end of the main script outside of the comments.
Fixed a potential loop when the end-user is IP authenticated to the Navigation Manager and a partial (incompletely rewritten) MNM request is made.
The URL part of the content attribute of the element meta http-equiv="refresh" which is generated directly in the DOM via JavaScript's document.write could have resulted in a possible wrong URL containing an extra ' in the end and in a JavaScript Syntax error due to a non-escaped in the cases where and " are not used interchangeably in JavaScript's document.write.
Rewrite URLs from the style attribute of dl... elements and from the href of the link rel="alternate stylesheet..." elements.
Muse Proxy HTML parser was sensitive to some locales and this is now fixed.
Corrected a regression bug related to the shutdown class that manifested on systems that have only private IP(s) assigned (e.g. 192.168.C.D) and the shut down process refused to send the shutdown command.

New in Muse Proxy 3.1 Build 02 (Apr 18, 2014)

New in Muse Proxy 3.1 Build 01 (Nov 6, 2013)

Implemented a PHP script that can be used for integrating in a portal the dynamically rewritten
links returned by Muse Proxy for the target Muse Proxy Sources accessed. The "6.5 Portal
Integration" section from the "${MUSE_HOME}/proxy/doc/Muse Proxy Advanced Con
figuration.pdf" document contains detailed information regarding Muse Proxy Portal Integration
and how this PHP script should be used in a portal.
Created the Anonymous application with the following features: - index page selects au
thentication method (IP or U/P); - each authentication group has its own sources group; - no
javascript or jquery; - pages are simple and with comments to easily identify each zone; - no GET
parameters, only POST; - "light" theme.
Added a new parameter "DELETE_CLIENT_SESSION_ON_LOGOUT", with possible values
true/false in the ${WEB_CONTEXT_HOME}/WEB-INF/web.xml file of the Administrator
Web Context and in the existing Muse Proxy Applications. This parameter tells the system
whether the Client Session must be deleted after a successful 'logout' action. If this field is missing,
the default value used will be 'false' meaning that the Client Session will not be deleted after a
successful 'logout' action.
Added the 'PROXY_USED' field in the Source's profile with the possible values: 'NO_PROXY',
SOURCE_LEVEL', 'APPLICATION_LEVEL', 'GLOBAL_LEVEL'. Depending on the value of
this field there will be used the proxy access details from the corresponding level. Added the
PROXY_HOST', 'PROXY_PORT', 'PROXY_PAC',
PROXY_AUTHORIZATION_USER_NAME',
PROXY_AUTHORIZATION_USER_PASSWORD' and
PROXY_AUTHORIZATION_SCHEME' parameters in the
WEB_CONTEXT_HOME}/WEB-INF/web.xml' file for Muse Proxy Applications. These
proxy access details will be used by a Muse Proxy Source when the 'PROXY_USED' field from
the Source's profiles has the 'APPLICATION_LEVEL' value. Previously, when a set of proxy
access details were set at global level and a Muse Proxy source did not used a proxy, all the HTTP requests done by the source did not used a proxy, but, when the rewritten 'Type 2' link was returned, it was chaining with the globally defined proxy. This was fixed and now if a source does not use a proxy then the rewritten link returned will not chain with a proxy either. Previously if a proxy pac returned a set of proxies and the first one of them failed, the Muse Proxy Source used the second one, but the navigation on the rewritten link was tried to be done using the first proxy returned by the proxy pac and the navigation failed. This was fixed and now all the proxies returned by the proxy pac which failed for the source will be ignored also when the navigation will be done on the rewritten link.
Previously, the JavaScript content included in the rewritten pages was computed internally in the Muse Navigation Manager code. Now the statical and dynamical parts of this JavaScript content are stored in 2 separate files and these files are included in mnm.jar at the build process. The dynamical part is updated with the run-time information before being appended to the JavaScript content.
Increased the Client Session Timeout value to 35 minutes. This value must be strictly greater than the Authentication Timeout for all of the existing Web Contexts. Also this value must be strictly greater than the Navigation Session Timeout. Added a new chapter named Muse Proxy Features in Muse Proxy.pdf that lists the features supported by Muse Proxy