CLC Genomics Workbench Changelog

What's new in CLC Genomics Workbench 6.5 Build 94986

• New features:
• Variant detection:
• New tool for adjusting read mappings through local realignment. The Local Realignment tool has the option to realign unaligned ends, realignment with a guidance variant track (e.g. obtained from external resources such as dbSNP, through the Indels and Structural Variants tool described below or from analysis of other read mappings) and allows for realignment of multiple samples. Has previously been available as a beta plugin.
• New tool for detecting structural variants (detects insertions and deletions, intra-chomosomal translocations, tandem duplications and inversions) working on "unaligned ends (soft clippings)". Has previously been available as a beta plugin.
• Important changes to variant reporting: adjacent variants are now reported as one variant instead of linked variants.
• A new variant filter has been added to both “Probabilistic Variant Detection” and “Quality-based Variant Detection”: “Ignore variants in non-specific regions”. This new filter ensures that variants in regions covered by just a few non-specific reads are ignored.
• Probabilistic Variant Detection: A new threshold filter, “Required variant count”, has been added to the wizard. This filter ensures that only variants present in a number of reads that exceeds the specified threshold are called.
• Quality-based Variant Detection: Addition of a new column that reports hyper-allelic status of variants. This is based on the specified threshold “Maximum expected allele” in the “Set genome information” wizard under “Ploidy”. The output in the table is “Yes” or “No” with respect to whether the threshold has been exceeded.
• A new column has been added to the variant track table that describes the length of the insertions, deletions, and replacements. This makes it possible to filter on the length of e.g. insertions/deletions.
• VCF export is now using genotype fields. The tag CLCAD is used for count of a variant, and PL is used for coverage. In this version, one variant track will result in one VCF file.
• Variant annotation:
• New tool for comparing variants between two samples
• Filter against known variants and Annotate from known variants: An MNV in the input track can be annotated as a partial match of an SNV in the track of known variants, if the SNV is a subset of the MNV.
• Filter against known variants: There is a new option to let MNVs be annotated as an exact match if several SNVs in the track of known variants can be joined to represent the full MNV allele sequence input track.
• When running the “Annotate with overlap information” tool using an annotation track as input and a variant track as parameter track, the column describing the specific variant in the Track Table now shows the position and description of the variants. The variant description also appears in the track tooltips when holding the mouse over the variants.
• Workflows:
• Automatic adjustment of layout in workflows. It is now (again) possible to adjust the connected workflow elements automatically. Right click in the workflow editor to access a menu with the option "Layout". Clicking on "Layout" will adjust the layout of the workflow. The layout can also be adjusted with the quick command Shift + Alt + L.
• Automatic update of tools in workflows. Tools in existing workflows will automatically be updated when opened from the Navigation Area. If new parameters have been added to the updated version of the tool, these will be used with their default settings. A workflow can be kept in its original form by saving the updated workflow with a new name as this will ensure that the old workflow is kept rather than being overwritten.
• Information: In the “Manage Workflows” dialog a new tab has been added providing information about the workflow (such as when it was built and which version of the workbench was used).
• Highlight used elements: In the Side Panel under “View mode” it is now possible to select “Highlight used elements”, which will show all elements that are used in the workflow. Unused elements are grayed out. The “Highlight used elements” can also be activated with the quick command Alt+ Shift + U.
• Highlight Subsequent Path: Is a further development of the old option “Mark Subsequent Path”. If you right click on the name of one of the tools in a workflow, it is possible to select “Highlight Subsequent Path”, which will highlight the path in the workflow from the tool that was clicked on and further downstream. All other elements in the workflow will be grayed out.
• Create Installer: A workflow can now be installed directly from the workbench. This can be done with the “Create Installer” button (or the quick command Alt + Shift + I). Three options exist in the “Create Installer” dialog: 1) Install the workflow on your local computer, 2) Install the workflow on the current server (requires that you are logged on to the server and that you are the administrator), and 3) Create an installer file to install it on another computer.
• Export can now be part of workflows.
• Workflow enabled elements can be dragged directly from the Toolbox into the workflow editor.
• Workflow images can be copied from the editor by using Ctrl + C (copy) and pasted into the desired destination with the Ctrl + V command.
• All elements can be removed from the workflow with the shortcut Alt + Shift +R.
• Previously, when running the “ChIP-Seq Analysis” tool, the result would be a copy of the read mapping with annotations added. Now the annotations are added to the read mapping used as input. Workflows using the "ChIP-Seq Analysis" tool must be manually updated, deleting the ChIP-Seq workflow element and adding it again.
• Reinstallation of modified workflows can now be done directly with the “Create Installer” function. A pop-up dialog provides the option to make "forced import" of an already installed workflow.
• Speed improvements in the workflow editor means that the user experience when editing large workflows has been greatly improved.
• New tools that are now workflow-enabled:
• Classical Sequence Analysis, Alignments and Trees
• Create Tree
• Maximum Likelihood Phylogeny
• Classical Sequence Analysis, General Sequence Analysis
• Extract Annotations
• Classical Sequence Analysis, Nucleotide Analysis
• Reverse Complement Sequence
• Reverse Sequence
• Molecular Biology tools, Sequencing Data Analysis
• Assemble Sequences
• Assemble Sequences to Reference
• Secondary Peak Calling
• Track Tools, Annotate and Filter
• Extract Reads Based on Overlap
• Track Tools, Graphs
• Identify Graph Threshold Areas
• Resequencing Analysis, Compare Variants
• Compare Sample Variant Tracks
• Transcriptomics Analysis, General Plots
• Create Histogram
• De Novo Sequencing
• Map Reads to Contigs
• 3D Molecule Viewing: The integrated viewer of structure files, the 3D Molecule Viewer, has been completely redesigned. The Molecule Viewer offers a range of tools for inspection and visualization of proteins and other molecules stored in structure files from the Protein Data Bank (PDB).
• De novo assembly:
• New tool: Map Reads to Contigs. This tool allows mapping of reads to contigs. This can be relevant in situations where contigs have been imported from an external source, the output from a de novo assembly is contigs with no read mapping, or if you wish to map a new set of reads or a subset of reads to the contigs.
• Scaffolds can be exported in AGP format: scaffolded contigs are exported as individual contigs and not as a single scaffold with N's inserted in between contigs. This allows for submission-ready data.
• Great performance improvement when updating the contig sequence based on reads that are mapped back to contigs.
• Tracks: Several new features have been added. Tt is now possible to:
• When there are more reads than can be shown in the available view area, an overflow graph will be displayed below the reads. The overflow graph was previously shown in grey. Now the overflow graph is shown in the same colors as the sequences. Hence, it is now possible to distinguish forward, reverse and paired reads in the overflow graph as well as mismatches in reads.
• Insertions from variant tracks and reads tracks can now be shown in tracks.
• For variant tracks, a new side-panel option “Insertion” allows the user to select whether to display insertions or not.
• For reads tracks insertions seen in more than a given percentage of reads are shown. The default percentage is 1%, setting it to 0% will show all insertions (like the cluster editor) and setting it to 100% will show no insertions.
• Insertions in reads tracks that are present at a frequency below the specified threshold are shown with a vertical line in the reads to indicate its location.
• Reads tracks now also have a mouse-over tooltip that provides information about insertions at specific positions. This tooltip reports the number of reads that contain the insertion and lists what was inserted.
• Extract reads from read tracks in two different ways:
• Extract sequences from tracks. Allows extraction of all reads as single sequences or as sequence lists.
• Extract from selection. Allows the creation of a reads track containing only reads that fall within the selected region, and of specific types.
• Four new options are available in the Side Panel for Track layout when viewing a reads track:
• Show quality scores: Makes it possible to adjust the colors of the residues based on their quality scores. In cases where no quality scores are available, blue (the color normally used for residues with a low quality score) is used as default color for such residues.
• Matching residues as dots: Replaces matching residues with dots in reads tracks. This option makes it easier to spot variants.
• Show read type specific coverage: When enabled, the coverage graph that summarizes those reads that could not be explicitly shown is now replaced by one coverage graph for each read type found in the Reads track. This can be used for easy and visual comparison of the strand specific coverage.
• Only show coverage graph: When enabled, only the coverage graph is shown and no reads are shown.
• A new tool has been included: “Identify Graph Threshold Areas”. This tool uses graph tracks as input to identify graph regions that fall within certain limits (thresholds that have been specified by the user).
• Extract annotations from track. This tool makes it very easy to extract parts of a sequence (or several sequences) based on its annotations.
• Create a track list with the shortcut Ctrl + L
• The create histogram tool now also accepts graph tracks as input.
• The error message "Too much data for rendering. Either zoom in to view data, or adjust data aggregation threshold" has now been added to the big grey box that appears in cases where a track cannot be shown. Previously only a big grey box was shown with no further explanation.
• Opening a large table view of a variant track is no longer blocking the user interface. It is running in the background, and it is possible to stop loading the data by closing the table view.
• The Coverage analysis tool is a new tool that can find regions in a read mapping where the coverage is suddenly dropping or rising.
• The "Assemble Sequences" and "Assemble Sequences to Reference" tools are now batch, server and workflow enabled.
• Assemble Sequences: Trimming is no longer integrated with the “Assemble Sequences” tool. This means that trimming must be done separately with the “Trim Sequences” tool.
• Export framework redesigned:
• Export of multiple files: you can export several files in one go. The naming of the file will default to the name used in the Navigation Area of the Workbench, but the user can specify a naming pattern to use instead.
• Export formats: A new column “Exports selected” has been added to the “Select exporter” table that indicates with a “Yes”, “No” or “Partly” whether the currently selected element can be exported with the given exporters. Partly means that you have made a selection of elements where only some of them can be exported by the selected exporter.
• Improved usability with a quick-select dialog for choosing the right export format. The dialog includes a description of each exporter that can be quickly filtered.
• Export can be integrated into workflows
• Support for direct compression of exported files in zip and gzip.
• Previously, VCF export required the user to know that both a variant track and a sequence track should be selected before exporting. This has changed, so that the user only has to select the variant track as input, and the sequence track is supplied as a parameter. This means it is more obvious that it should be selected, and it also means that the choice of sequence track will be remembered for the next vcf export.
• The folder viewhas been improved with the following:
• It is now possible to drag and drop objects from the folder editor. This will create a copy of the objects at the selected destination.
• Attribute columns will be left empty if the attribute has not been defined (previously attribute values that had not been defined were set to 0 and checkboxes were shown as unchecked).
• A new column showing the first 50 residues of each sequence has been added as an option.
• The column with the name “Length” has been renamed to “Size”.
• The column “Size” shows the length of a sequence, or for sequence lists, the number of sequences e.g.:
• Sequence or contig lists: the number of sequences/contigs
• Read mappings: the length of the consensus sequence
• De novo assemblies: the length of the reference
• Alignments: the length of the alignment
• The Side Panel “Save/Restore Settings” function has been expanded with a new feature:
• The “Save/Restore Settings” function (found at the top of the Side Panel) has been redesigned. It is now possible to save settings in two different ways. 1) The settings can be saved for this element type in general, e.g. for a track it would be save settings “For Track View in General”. In this way the settings will be applied each time you open an element of the same type, which in this case means each time one of the saved tracks are opened from the Navigation Area, these settings will be applied. These “general” settings are user specific and will not be saved with or exported with the element. 2) Settings can be saved with the specific element only e.g. for a track it would be save settings “On This Track Only”. The settings are saved with only this element (and will be exported with the element if you later select to export the element to another destination).
• Alignments: If you have one particular sequence that you would like to use as a reference sequence, it can be useful to move this to the top. This can now be done automatically by right clicking on the sequence name and then selecting “Move Sequence to Top”.
• The Sequence List Table has been improved with a new feature. A new column showing the first 50 residues of each sequence has been added as an option.
• SOLiD import now accepts XSQ files
• The following Plug-ins are now fully integrated in the Workbench:
• InDels and Structural Variation (old plugin name: "Structural Variation")
• Local Realignment
• Extract Annotations
• The tomato genome, Solanum lycopersicum SL2.40.18, available in the Download Genome tool.
• Phylogenetic trees:
• Create Tree now support the Kimura 2-parameter substitution model for DNA sequences and Kimura's distance estimate for protein sequences (Kimura 1983).
• It is now possible to construct Maximum Likelihood phylogenies from protein sequences.
• Improvements:
• Scrolling in read mappings: The mouse scroll speed through read mappings can now be performed with increased speed. Shift + Alt + Mouse wheel scroll makes the scroll 10x as fast as when using Alt + Mouse wheel scroll. When zoomed all the way in, each mouse wheel step scrolls 10 rows.
• Sort Sequences by Name: The multiplexing tool now allows a delimiter between group names in the “Sort Sequences by Name” wizard. This means that the selected group names are separated by an underscore. Previously all selected group names were merged without any delimiter.
• Cloning: The cloning editor can now work without having a designated vector. In essence this means that when no vector is selected you go directly in “Stitch mode” when a fragment has been selected, whereas you go in “Cloning mode” when a cloning vector and a fragment are selected.
• Renaming of data in the Navigation Area by clicking twice has been improved. Previously, you could accidentally enter rename mode when the intention was to get focus in the Navigation Area. Now, you can only trigger rename by clicking when the Navigator has focus.
• Filter Annotations on Name: The wizard layouts for the tool when used directly as opposed to when included in a workflow has been standardized.
• Extract consensus sequence tool:
• It is now possible to use the quality scores when resolving conflicts or disagreements between reads with “Insert ambiguity codes”. Previously, “Use quality scores” could only be selected when using the “Vote” option for conflict resolution.
• Low coverage regions are now annotated in the consensus sequence produced.
• The Extract Consensus Sequence dialog is now shown when extracting the consensus sequence when right-clicking a selection on the reference sequence in the mapping view, enabling the user to extract part of the consensus sequence.
• The Extract Consensus Sequence dialog is now shown when extracting the consensus sequence when right-clicking the name of the consensus or reference sequence, or when clicking the Extract Consensus button in the mapping table. The right-click menu option on the consensus to Open Sequence Including Gaps has been removed, since this functionality is now covered by the Extract Consensus Sequence tool.
• When using the “Translate to protein” tool, the max limit has been raised to 1GB.
• The sequence action "Open Copy" has been removed and "Open This Sequence" has been renamed to "Open Sequence".
• The alignment tool is now more memory efficient.
• Tables: Improved auto-adjustment of the column width (based on content and number of columns).
• Read mapping: The speed of running a read mapping against a masked reference has been improved significantly. When mapping reads to a reference sequence, it is possible to map reads to only selected annotated regions of the reference (= masking). Previously masking of a reference was performed by replacing the masked out nucleotides with N's. The new masking method discards the masked out nucleotides by splitting the reference into separate sequences. Hence, the masked out sequences are completely ignored in the analysis. The remaining sequence fragments are positioned according to the original unmasked reference sequence.
• Read mapping: The status bar in the lower right corner now shows the corresponding positions on the reference/contig sequence.
• The read mapper will now place ambiguous gaps to the left, as opposed to the right, to ensure better concordance with common variant databases.
• BLAST has been upgraded to BLAST+ 2.2.28 that includes a number of improvements and bug fixes. A full list of BLAST+ 2.2.28 changes can be viewed at http://www.ncbi.nlm.nih.gov/books/NBK131777.
• Usability improvement of simple table filtering:
• A dedicated filter button has been added to apply the filter directly without having to wait until the filter is automatically applied
• For tables with more than 10000 rows, the filter will not be applied automatically after a delay. Instead, there is a helping text asking the user to apply the filter through the "Filter" button. This avoids premature filtering before entry of the filter text has completed. Since filter can take some seconds for large tables, this used to be an annoyance because the user would have to wait until filtering was done to complete the entry.
• Phylogenetic trees:
• Bootstrapping with the "Maximum Likelihood Phylogeny" is now possible.
• Bootstrap values are now displayed in percent instead of absolute numbers.
• Bug fixes:
• Numbering of amino acids when calculating amino acid changes was wrong for coding regions spanning the starting point of circular chromosomes. We recommend running amino acid calculation again. Please note that the actual amino acid change is called correctly, only the numbering is affected.
• PDF export of the history of a result did not include the name and version number of the Workbench that produced the result.
• Phylogenetic trees:
• The Juke-Cantor distance estimate now ignore all positions containing gaps in pairwise alignments.
• Disabled substitution rate estimation when the corresponding option is deselected by the user in the Maximum Likelihood Phylogeny tool.
• Fixed a bug that caused branch lengths to be estimated incorrectly for ML trees.