Picalo is a data analysis application, with focus in fraud detection and data retrieved from corporate databases. Picalo is also the foundation for an automated fraud detection system (see below).
Picalo is currently focused on data analysis for fraud and corruption detection. However, it is an open framework that could actually be used for many different types of data analysis: network logs, any type of database-oriented data, scientific data and
Picalo is built upon a three-level architecture, including open source and potentially closed source parts.
Detectlets are one of the most exciting parts of the Picalo architecture. They allow non-programmers to run analysis routines created by others. See the detectlets page for more information.
Picalo is built upon the shoulders of many great projects. Thousands of individuals have contributed time and energy to these projects, and the Picalo effort is grateful for their work. These are listed as follows:
· Python
· wxPython
· mxDateTime
· Python Windows Extensions
· Statistics package pstat.py by Gary Strangman
· Nuvola Icon Set by David Vignoni
· Gadfly Python Database
· Matplotlib
What about quality control? Another way to phrase this question is, "Can I trust Picalo's results?" The short answer is you can trust Picalo as much as any analysis application. We take quality control very seriously.
The long answer is you should never fully trust any analysis application. You should always double check each step of the process, print control totals, and manually ensure that your routines are doing what you think they are. It can be very embarrasing (and dangerous) to make decisions on faulty analysis routines.
Users unfamiliar with the open source world may implicitly trust "corporate" software and be wary of "open" software. We hope you will re-evaluate this common misconception as you use Picalo.
Certainly, closed-source software applications are often more user friendly than community-built applications. But "good looking" and easy-to-use programs are not necessarily trustworthy.
As more users test Picalo and more developers help program it, we'll have a lot of eyes looking through the code and testing the routines. Open source software often finds and fixes bugs much faster than closed-source software because of the number of individuals looking at its code. "Corporate" software is often written by small development teams who are driven by marketing calendars and new features.
The open source world has many examples of incredibly well-written software, including Linux (widely known for crashing very rarely), PostgreSQL (a highly-respected database), Apache (which runs most of the web), Bind (which runs the domain names on the Internet), KDE and Gnome (excellent user interfaces that look similar to Windows), wxWidgets (the GUI toolkit Picalo is built upon), Perl and Python and GCC (programming languages many "real" programs are written with), LaTeX (a great word processing platform), and Firefox (an incredible web browser). This list could go on with thousands of successful open source products that are in production use today.
In summary, is Picalo perfect? Of course not. There may even be one or two bugs left in the Level 1 routines of Picalo. But we're working to make it a world-class analysis application that encodes information about thousands of fraud schemes used worldwide. Consider helping out to be part of the team that makes Picalo better and better.
What's New in This Release: [ read full changelog ]
· Changed delimited text to save with the line separator of the platform (Win or Unix)
· Removed all use of the python 'csv' module because it took too much memory.
· Added a new picalo.lib.delimitedtext module to do csv, tsv, and other delimited text formats. It is slower than the built-in, C-based csv module, but it can handle millions of records. I tested it against 50 million records (4 GB file) and it never used much memory.
· Fixed delimited and fixed reading and writing to handle unicode correctly. Unicode has long been supported by Picalo, but some encodings were fragile. It works robustly now.
