Patterns of the Void

QAssert: JavaScript assertions with AJAX reporting!

GaB — Wed, 16 Nov 2011 16:54:51 +0000

QAssert is a powerful, easy-to-use, JavaScript assertion suite. In contrast to console.assert() it enables you to report failures via AJAX. It gives you also powerful assertion functions and is cross-browser compatible.
It can be used in both in unit, functional and manual tests, and notably also
in production code.

QAssert is especially useful for regression testing: You can place assertions for any assumption that is made in the JavaScript logic. While running the code assertion violations will be reported via AJAX and regressions become quickly visible. To give you an example how to safeguard against unexpected DOM changes:
var $panel = $("#menuPanel"); $panel.assert().load("/"); $.assert($panel.text().length);

It is a simple drop-in jQuery plugin. It is disabled by default. To enable it and to see assertions:
$.assertSetup("/assertReporter"); // Required only once. $.assert(false)
As a result:

You will se the assertion failure in the console including a full stack trace (Assertion failed: false undefined {anonymous}()@file:///D:/Dropbox/dev/qassert/qassert/qassert.js:708 …)
You will see an AJAX call being made to the given URL

It is highly configurable:
$.assertSetup({ enabled: true, ajax: "/fooBar", // You can also pass a full $.ajax() parameter object here! catchGlobalErrors: true, // This will report ALL uncatched exceptions too! contextCallback: function(value, message, stacktrace) {return {version: app.version, locale: app.locale}}, // Enhances the reported data log: myLogFunction, // Used to log failures locally, by default to the console. title: "Assertion sucked up:" });

It has a convenient API:

$.fn.assert(message, sizeOrCallback): Chained selector assertions.
$.assert(value, message): Boolean assertion.
$.assertDeepEquals(value, expected, message): Recursive equality assertion.
$.assertEmpty(value, message): Empty assertion.
$.assertEquals(value, expected, message): Equality assertion, no recursion.
$.assertIs(value, type, message): Type assertion.
$.assertNot(value, message): Boolean assertion for false.
$.assertNotDeepEquals(value, expected, message): Recursive unequality assertion.
$.assertNotEmpty(value, message): Non-empty assertion.
$.assertNotEquals(value, expected, message): Unequality assertion, no recursion.
$.assertNotIs(value, type, message): Type assertion inverted.
$.assertNotSame(value, expected, message): Strict unequality assertion.
$.assertSame(value, expected, message): Strict equality assertion.

See the documentation and test cases.
Join the development: https://github.com/gaboom/qassert

Instant PHP eval

GaB — Tue, 04 Oct 2011 22:49:25 +0000

Here is the code you get a form with, you can submit php code, it gets evaluated on the server.
SECURITY WARNING: command(s) below will be all executed in PHP’s server context! Deny public acces.



    
        
        PHP evaluator
    
    
        
            ";
                var_dump($result);
                echo "";
                print_r($result);
                echo "";
                echo($result);
            }
            ?>
        
        
        SECURITY WARNING: command(s) below will be all executed in PHP's server context! Deny public acces.
        
            <?php echo $arguments ?>
            

            Function:

How to write down passwords onto paper safely?

GaB — Mon, 18 Jul 2011 20:46:26 +0000

Writing down usernames and passwords with a pen is insecure in so many ways I do not attempt to count.
The sheer count of passwords however, forces most persons to store them somewhere, instead of keeping all of them just in your head.
The digital solution is to protect the passwords with another password in sort of a vault (e.g. in a file on a password protected disk, encrypted volume or file, password storage program) [Please note: browsers store passwords almost cleartext by default!]

Sometimes you are still forced to keep a password written up quick and easily in an unencrypted environment, like a piece of paper. (Developing and memorizing secret code language is rather difficult to accomplish this.)
The trick is to write down the password, but with a simple to remember twist!

For each password pair you can see first an example of a cleartext password, then the transformed version:

alaudZAPP1 – alXZaudZAPP12
persepoleS – pe9lrsepoleSq
Bo23LOwf3 – BobT23LOwf34

You can insert at the very same position(s) random letters to make the text written down unusable. In this example I choose to insert two the after the first two letters, then one at the end.
Looking at the passwords on the right, remembering your scheme, you can easily re-type (decode) the original passwords but leaving an attacker with an obfuscated one.

WARNING: This method is by far not as safe as a fully encrypted password, and offers only a better-than-nothing protection. Common patterns in your password will become visible by this! Never re-use a password, this method reveals any such mistake, also making it obvious how you encoded all your other passwords written down.

Three questions for an architect

GaB — Fri, 15 Jul 2011 00:25:49 +0000

For a release sign-off

Do all unit, automated and regression tests run?
What is the status of outstanding bugs?
Are all changes demonstrated to stakeholders?

For a design sign-off

Are all transformations from the input domain to the output domain possible? Identify first risky and/or complex ones.
Are all our needs supported by an API?
Is our API supporting all requirements?
BONUS: If it takes more than two days to complete, does a task breakdown exist?

Business Software Development

GaB — Wed, 27 Apr 2011 21:46:45 +0000

Business Software Development
Eötvös Loránd University Budapest, Faculty of Informatics
Eötvös Collegium Informatics Workshop 2011-

Syllabus

What is business, software and development?
Goals, realization and measuring success. Sustaining a business.
Software lifecycle, development as a process.
Planning and executing projects: Lean, Agile, Scrum.
Architecture, design and documentation.
Requirements: functional and non functional. User interaction and interface design.
Version control and configuration management. Integration and release.
Testing and bug tracking, support and customer awareness.
Understanding different company cultures and roles.
Copyright and intellectual property.

Literature

Real Software Engineering [Glenn Vanderburg; Lone Star Ruby Conference; 2010]

Code as Design [Jack W. Reeves; Developer.*; 1995]

No Silver Bullet: Essence and Accidents of Software Engineering [Frederick P. Brooks, Jr.; Computer Magazine; April 1987, University of North Carolina at Chapel Hill]

Design Patterns: Elements of Reusable Object-Oriented Software [Erich Gamma et al.; Addison-Wesley Professional; 1994; ISBN: 978-0201633610]

The Mythical Man-Month: Essays on Software Engineering [Frederick P. Brooks; Addison-Wesley Professional; 1995; Anniversary edition; ISBN: 978-0201835953]

The Pragmatic Programmer [Andrew Hunt, David Thomas; Addison-Wesley Professional; 1999; ISBN: 978-0201616224]

How to archive Drupal as a static site?

GaB — Sun, 27 Mar 2011 19:01:35 +0000

Here is the summary how I’ve managed to archive my Drupal weblog before migrating to WordPress. To do this, you should never put your blog/site to the root of your URL, i.e. example.com/ .
Instead, put it under a directory like example.com/foo so you can it archive it under this address and put up your new site at example.com/bar . The root address should always redirect to the current site.
Thanks to http://drupal.org/node/27882 and http://www.starbowconsulting.com/blog/tao/replacing-live-drupal-site-static-archive .

Steps to make the site ready being archived:

Disabled all commenting with update node set comment = '1'
For each content-type, set the comment controls to “Do not display.”
Disable search completely, also in the theme
Remove all dynamic blocks, login, register, who’s online, actual links, feed links etc.
Put up a static block and a sticky post stating this is a static archive with a link to the new one.
Remove all spam comments and spam links before archiving or search engines will ignore you.

Then I used httrack to archive my site:
httrack http://example.com/foo -W -O "~/static_archive" -%v --robots=0

That’s it. Double check your logs, I’ve had some old nodes giving a server error because of an outdated plugin.
Now you can remove your Drupal site and drop the corresponding tables. But there is one thing left:
If you used pretty URLs, like /node/4, you will need to set up redirects to show the archived html files instead (/node/4.html).
You might also want to redirect the RSS feeds to migrate your RSS subscribers to your new site and its feed.

RewriteEngine on # If we are accessing root, redirect to index.html RewriteCond %{REQUEST_URI} ^/foo/$ RewriteRule ^(.*)$ index.html [L] # Redirect RSS feeds RewriteCond %{REQUEST_URI} ^/foo/rss.xml$ RewriteRule ^(.*)$ http://example.com/foo/feed/ [L] RewriteCond %{REQUEST_URI} /foo/feed$ RewriteRule ^(.*)$ http://example.com/foo/feed/ [L] RewriteCond %{REQUEST_URI} /foo/.*/feed$ RewriteRule ^(.*)$ http://example.com/foo/feed/ [L] # If we are accessing a page without a "." in it, append .html to the end. RewriteCond %{REQUEST_URI} !\. RewriteRule ^(.*)$ $1.html

Import XLS/CSV into MS SQL

GaB — Mon, 18 Oct 2010 12:44:04 +0000

What about the simple task of importing a tabular Microsoft Excel Sheet into a Microsoft Transact SQL table?
After paying thousands of dollars in license fees, this will take you a day! And you will need OpenOffice to do that :)

First google “import xls into mssql” will give you a simple solution:

SELECT * INTO XLImport5 FROM OPENROWSET('Microsoft.Jet.OLEDB.4.0',
'Excel 8.0;Database=C:\test\xltest.xls', 'SELECT * FROM [Customers$]')

That will not work! Jet provider is not supported on 64bit! Yeah, Microsoft is not supporting 64bit users of its enterprise database solution. Thank you very much. (Just google how many ppl are ranting about the lack of this…)

OK! No problem I’m going to write a program that generates INSERT INTO statements from a CSV export, here you go:

package eu.gablog.csv2sql;

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.text.DecimalFormat;
import java.text.DecimalFormatSymbols;
import java.text.ParseException;
import java.util.LinkedList;
import java.util.List;

import au.com.bytecode.opencsv.CSVReader;

public class CSV2SQL {

    private static final boolean truncate = false;
    private static final String src = "D:\\tmp\\src.csv";
    private static final String dst = "D:\\tmp\\dst.sql";
    //private static final String DB = "HCOM_EMEA.dbo.HCOM_BOOKING_POSTPAY_MAPPING";
    private static final String DB = "MHOTEL.dbo.PROPERTY";

    // GENERIC VERSION: first row contains column names
    public static void main(String[] args) throws IOException {
        // PARSE ALL ROWS
        CSVReader csvRows = new CSVReader(new FileReader(src), ';', '"', 0);
        String[] nextLine;
        List rows = new LinkedList();
        while ((nextLine = csvRows.readNext()) != null) {
            rows.add(nextLine);
        }

        // PREPARE OUTPUT SQL
        BufferedWriter out = new BufferedWriter(new FileWriter(new File(dst), false));
        if (truncate) {
            out.write("DELETE FROM " + DB + ";");
            out.newLine();
        }

        // INSERT STATMENT
        int rowCount = 0;
        String insertInto = "";
        for (String[] row : rows) {
            if (rowCount == 0) {
                insertInto = parseColumns(row);
            } else {
                String insert = parseRow(row);
                out.write(insertInto + " VALUES " + insert + ";");
                out.newLine();
            }
            rowCount++;
        }
        out.close();
        System.out.println("DONE: " + rowCount + " lines parsed from " + src + " to " + dst);
    }

    private static String parseColumns(String[] columns) throws IOException {
        StringBuilder result = new StringBuilder();
        result.append("INSERT INTO " + DB + " (");
        boolean first = true;
        for (String column : columns) {
            if (!first) {
                result.append(", ");
            }
            result.append("[" + column + "]");
            first = false;
        }
        result.append(")");
        return result.toString();
    }

    private static String parseRow(String[] row) throws IOException {
        StringBuilder result = new StringBuilder();
        result.append("(");
        boolean first = true;
        Double d;
        for (String column : row) {
            if (!first) {
                result.append(", ");
            }
            if (column.isEmpty()) {
                column = "NULL";
            }
            if ("NULL".equalsIgnoreCase(column)) {
                // NULL
                result.append(column);
            } else if (isLong(column) != null) {
                // LONG
                result.append(column);
            } else if ((d = isDouble(column)) != null) {
                // DOUBLE
                result.append(d.toString());
            } else {
                // STRING VALUE
                result.append("'" + escape(column) + "'");
            }
            first = false;
        }
        result.append(")");
        return result.toString();
    }

    private static Long isLong(String str) {
        try {
            return Long.parseLong(str);
        } catch (NumberFormatException nfe) {
            return null;
        }
    }

    private static Double isDouble(String str) {
        try {
            DecimalFormat df = new DecimalFormat();
            DecimalFormatSymbols dfs = new DecimalFormatSymbols();
            dfs.setDecimalSeparator(',');
            df.setDecimalFormatSymbols(dfs);
            Double d = df.parse(str).doubleValue();
            if (!d.toString().replace('.', ',').equals(str)) {
                d = null;
            }
            return d;
        } catch (ParseException e) {
            return null;
        }
    }

    private static String escape(String str) {
        String result;
        result = str.replaceAll("'", "''");
        return result;
    }

}

You think you can use that just this easily? Of course not! Microsoft Excel does not know anything about UTF-8 in 2010 AD!
CSV export has some random encoding just to fuck with you for the fun. No option for having it in UTF-8.
Just fire up OpenOffice, go to Sava As, and enjoy the beautiful options of selecting the export’s encoding. Now use the converter on this input.

Captcha for CakePHP 1.3 with KCAPTCHA

GaB — Wed, 08 Sep 2010 12:49:41 +0000

We are going to use KCAPTCHA to generate Captchas and verify them in form submissions in CakePHP. Look at phpinfo(), you will need

First, download KCAPTCHA and put the directory “kcaptcha” from the .zip into app/vendors/.

Insert this into any view you want to have a captcha on:



        
    

input('captcha', array('label' => __('Nem vagyok robot: 
', true))); ?>

We load the image with a semi-random URL, ensuring to always load a new one. In my case, it will be loaded from AjaxController:

function captcha($random = null) {
        $this->layout = null;
        $thiy->$autoRender = false;
        //$this->RequestHandler->setContent('jpeg', 'image/jpeg'); // use this if your mime-type is incorrect

        App::import('Vendor', 'kcaptcha/kcaptcha');
        $kcaptcha = new KCAPTCHA(); // renders the captcha image fully here
        $this->Session->write('captcha', $kcaptcha->getKeyString());
    }

As you can see, displaying a captcha by fetching it, immediately stores the captcha’s solution in the ‘captcha’ session variable.

When you process the submitted form, where you have included the captcha, use this (Component) function to check the submitted value against the one stored in the session.

function checkCaptcha($text) {
        $captcha = $this->Session->read("captcha");
        $this->Session->delete("captcha"); // SECURITY: must clear captcha to avoid repatcha
        if (empty($captcha) || $captcha != $text) {
            return false;
        }
        return true;
    }

Be aware, that upon checking, regardless of the match, the captcha must be destroyed (removed from the session). Otherwise recaptcha is possible (re-submission over and over again with a first valid captcha).

At last, kcaptcha_config.php to finetune your images.

Twitter OAUTH

GaB — Fri, 02 Jul 2010 00:13:53 +0000

Twitter will phase out the widely-used HTTP basic authentication by August 2010. The amount of working examples online makes me believe that most applications that make use of the Twitter API is not migrated yet. Even the simplest action will requires to use OAUTH.

This tutorial will guide you through the authentication process. In the end of our example we will get hold four secret keys in our hands that can be used at any later time to access the API with the same credentials again. You can download all files here. (You’ll need CURL to be enabled in PHP.)
The walk through corresponds to the “Desctop Clients” scenario in the first link.

1. Go to http://twitter.com/apps/ and click “Register new application”. For Application Type choose Browser and for the callback URL, the URL of confirm.php.
Now you receive a Consumer Key and a Consumer Token. Put them into secret.php.

2. Fire up index.php. You can double check your Consumer credentials here. First you’ll see only one link to authorize via twitter. Follow that, confirm the access.

3. Twitter’ll redirect you back to confirm.php. But you also gained now the Provider Token and Secret! Put them as printed on the screen into secret.php.

4. Going back to index.php you’ll see that you have no all the keys you need. You can check it on random.php, that uses them. Store this four keys securely. They won’t be revoked, unless the twitter profile explicitly removes the app. It’ll work even after changing the password.

Having just this four keys, using the same API classes, updating the status is as easy as this:

twitterObj = new EpiTwitter($this->consumer_key, $this->consumer_secret, $this->provider_token, $this->provider_secret);
    }

    /** Tweets the message with '...' shortening. If $url given, appends it.
     *
     * @param  $message
     * @param  $url
     */
    function tweet($message, $url = null) {
        $length = $this->maxLength - (empty($url) ? 0 : mb_strlen($url)+1);
        if (mb_strlen($message)>$length) {
            $length -= 3;
            $fix = "...";
        } else $fix = "";
        $message = mb_strcut($message, 0, $length).$fix;
        if (!empty($url)) {
            $message .= " ".$url;
        }
        return $this->twitterObj->post_statusesUpdate(array('status' => $message));
    }

}
?>

If you don’t store the Provider’s token and secret, instead storing it in the session, you can use Twitter accounts to authenticate profiles on your site.

Hello world!

GaB — Mon, 21 Jun 2010 21:46:50 +0000

It’s about time. Finally I managed to migrate my blog engine to WordPress instead of Drupal.

This is mainly a technical decision:

Blog engines regularly receive software updates. Updating Drupal is a real royal pain in the ass. Every two weeks you need to disable your site, disable all plugins, unzip the core and all plugins individually, copy them into place, run update tasks on the database, enable site, fix broken plugins… I prefer just to click on update in the admin panel of WordPress.
Input filters in Drupal allow you to configure all sort of orders, priorities, exceptions and more. Unfortunately I never found a combination where all these filters work together not messing up included source code in my posts.
Druapl is a heavyweight platform for arbitrary web applications, community pages, forums etc. I need only simple blog engine.
WordPress has all the customization I need, its admin panel even allows me to edit any plugin code in the browser. But once configured I do not want to spend any time on updates and integration.

New posts will be available at http://gablog.eu/void. Unfortunately I can’t migrate my posts. I’ll make a static page out of what I have in Drupal, available at the original URL: http://gablog.eu/online.