November 25, 2010

Garbage Collector

One existential part of Network Operation System (NOS) is [distributed] memory manager a.k.a. Garbage Collector. If we think about (G)lobality of GC , and continuations of task mapping to nodes, and also habits of NOS users we can expect that GC will can guided  by Celestial mechanics (Internet works this way).


Idea is to trigger Processor level GC by daylight line. 

Let first mark-and-sweep happened in Tokyo when Sun risen up. Guys coming to local Stock Exchange will push some analytic to NOS for processing. Then NOS will try to distribute algorithm to data locations in US and Russia where lots of idle nodes at night time.

Then it is Beijing turn to sweep their nodes and so on around.




November 17, 2010

GREP evolution

$ grep -r `some
^C
Why doesn't it find `some in files recursively?  Is it example of bad user inteface?

In fact it is not that ad-hock as it seems. One just need bit of UNIX cultural background to come to it logically.

Fact one is that grep is in fact  g/re/p (go with regexp and print) command in VI editor. It's command line version  have to be lifted over list of files and when mapped over directory tree have to use  "find . | xargs grep `some" instead. Eventually somebody made it script - rgrep.sh and other guy integrate it to grep as -r option.

find . -| xargs vi -e g/some/p 



November 8, 2010

Node.js - Node.os

Hi,


Node.js is just `processor and any `job can be `assigned to it. JavaScript is `system language for defining certain `computation and executing it as `job. Job 's has to be `synchronized over nodes to produce result of computation.

This is, basically, definition of NETWORKING OS, where:


  • Processor - node capable of executing Job.
  • Job - scheduled unit of computation.
  • System language - describes computation in terms of OS primitives, primitive computations and data-flows.
  • Synchronization - suppose tasks {A, B}, with I(nput) and O(utput). 


Possible synchronizations are:
1)
AO -> BI (A-Out implies B-Input) - This is PIPE like synchronization for sequence of tasks.

The OS primitive for it (see Algebra of Communicating Processes) is '.' - B.A - B after A.

Going further we can introduce `+' - choice and `|' - interleaving.

Execution of jobs in MapReduce process synchronized on inputs and intermediate outputs on node level and cluster level.

(AI -> BI -> ... -> ZI) - This synchronization characterize Map phase, which also taking on Reduce of outputs: (AO -> BO -> ... -> ZO).

Those definitions calls for POSIX like specification which may serve as solid base for 'Services (aka SaaS) design.


November 5, 2010

Continuous Integration UNIX way

Hi, there was a time I couldn't write SHELL scripts. For example, once I had to confirm 1000+ file deletion by pressing 'y'. Now I would do > yes | rm /1000/1/night/* ..., but then I used screwdriver, sticking it in between Y and H.



Personally, I can't imagine any development setup on UNIX without Emacs. Probably, VI would do but it's internal language far inferior to emacs-lisp.

So, here we go again - my desktop, that did not change for 3+ month...



It Emacs on the left and something what missing from Emacs functionality on the right (otherwise it wouldn't be there:)).

While developing significant piece of software we often feel insecure. It's especially true for dynamic languages which gives no guaranty of any kind about runtime (they are too rock-n-roll IMO, some type inference wouldn't harm).
To leverage this feeling we adding amount of bravado into SW life-circle with Maven, Hudson, Agile and Continuous Integration/Deployment. It helps, but seems like overkill for me and while this Zen can be practical in corporate aquarium (because they have office plankton there) it is not for individual projects (when no Zen/Lean master around). 

Here is my receipt for better sleep while still have fun with big amount of code. 

REM We would need nothing but linux-base to organize relevant TDD/CI environment for development with JavaScript.


For Integration Test I would recommend JsTestDriver from Google. It is quite sane piece of software with sound integration test-strategy.
We need to start server, register target browser's as test slave and trigger tests on repository when ready.

Those bits of manual labor can be automated with Bash script.

run_jsTestDrive.sh
# test-runner Continuous Integration style 

java -jar $JSTESTDRIVER_HOME/JsTestDriver-1.2.2.jar --port 4224 --captureConsole &

PID=$!

trap "{ echo 'Exiting'; echo $PID;  kill $PID ; exit 0; }" SIGINT SIGTERM

sleep 3

while true;
do
      sleep 2

      java -jar $JSTESTDRIVER_HOME/JsTestDriver-1.2.2.jar --tests all | \
      tee >({ \
      grep -q -s 'Total.*Fails: 0; Errors: 0'; \
     
      if [ $? -eq 0 ]; then \
          echo -e "\e[1;32m Tests successful - Fails: 0; Errors: 0  \e[0m"; \
      else \
          echo -e "\e[1;31m Tests failed. \e[0m"; \
      fi \
      })

      inotifywait -e modify *.js

done


It will start (and kill) server for us. It will look on disk for changing files and fire tests if any.

NB. It's very important that all your test have colored diagnostic. It doesn't meter what it says but it must be GREEN or RED for your peripheral vision to pick up it easily.

Tests failed. vs. Tests successful - Fails: 0; Errors: 0 

Also colors are big importance because TDD process based on REFACTOR-GREEN-COMMIT cycle, and 'green means tested.

There is some mambo-jumbo in the script which does coloring with terminal control sequences.

File changes picked up by kernel inode supervisor using user land utility $ inotifywait -e modify *.js



For Unitary Test I had to write small piece.

There about dozen well written xTest frameworks but I wanted to keep same test database for `unitary and for `integration. While we already fixed CI framework to js-test-driver, its TestCase format have to be reused with new unitary TestRunner.

TestCase("js-test-unit", {
        
        setUp: function() {
            jstestunit.testCaseManager =
                new jstestunit.TestCaseManager();
            
            jstestunit.testCaseBuilder =
                new jstestunit.TestCaseBuilder(jstestunit.testCaseManager);
            
            jstestunit.testRunner =
                new jstestunit.TestRunner();
        },
        tearDown: function(){
            delete jstestunit.testCaseManager;
            delete jstestunit.testCaseBuilder;
            delete jstestunit.testRunner;
        },
        
        "test configuration exist": function(){
            
            assertEquals('object', typeof jstestunit.config);
            assertEquals('object', typeof jstestunit.testCaseManager);
            assertEquals('object', typeof jstestunit.testCaseBuilder);
            assertEquals('object', typeof jstestunit.testRunner);

        },
 });

Sources is in the repo on BitBucket: $ hg clone https://lib.aca55a@bitbucket.org/lib.aca55a/js-test-unit

This TestRunner is HTML infected and have to run directly inside Browser. To integrate it with Emacs we would need another module - MozRepl. This one is hooked directly in Gecko engine of FF and shows on localhost:4242. On Emacs side it is just sock to REPL.  From the MozRepl you can `BrowserReload() effectively re-executing all your tests.

Again we can automate stuff in emacs-lisp this time.

;; M-x moz-reload-mode 
(define-minor-mode moz-reload-mode
  "Moz Reload Minor Mode"
  nil " Reload" nil
  (if moz-reload-mode
      ;; Edit hook buffer-locally.
      (add-hook 'after-save-hook 'moz-reload nil t)
    (remove-hook 'after-save-hook 'moz-reload t)))

(defun moz-reload ()
    (moz-firefox-reload))

(defun moz-firefox-reload ()
  (comint-send-string (inferior-moz-process) "BrowserReload();"))


This setup did not add functionality. You REPL code normal way, occasionally saving files. But general feeling is like you hawing angel on right shoulder singing `Don't worry, be happy... .

Demon on left knows real state of affairs, and this what happens when you connection human to machine - you have to leverage fear.

1 Fr - define fear between zero and 1.0 software, when constant being-in-the-flow of 1 challenge applied to coding, produces a function of 1 hack, the programmer not being a seat of any interruptive force. 

Few tips in the end.

  • It is true that Emacs is best in the business but it is also true that it is hard to get on this 50+ year marked. I'm there today thanks to unknown-somebody who's config I `borrowed some years ago. Still use it - $ hg clone https://lib.aca55a@bitbucket.org/lib.aca55a/env . It is easy to discover key-binding in emacs-keys.el and module settings in emacs-packages.el for starter.
  • It is important to get right setup for you language. Examples of `right are:
  • There are integrated solution which easier to work with, but keep keep in mind that they raise fear to totally new level.
P.S. My testing harness runs outside of Emacs and when some error are in the report I have to communicate manually to editor. Still some automation goes in:
while read f; do emacsclient -n `echo $f|sed 's/\([^:]*\):\(.*\):.*/+\2 \1/'`; done


November 2, 2010

Inheritance models.

Hi, Here I keen to describe only practical models.


There is general properties of model which measures amount of information we can capture using this model.

Let say we describe some particular system using set of properties. Arrangements of properties constrained by inheritance model that we choose.

Models differ by redundancy it require (encapsulation vs. delegation) and reuse it provides (inheritance vs. inclusion).

Numerically we can capture redundancy by combinatory complexity but reuse can only feet in functor which doesn't have something like adjustency degree (? or does it).


Lets us look for model in the field of `modern` programming  languages.

Java diagonal inheritance model



There is two parallel hierarchies: one for interfaces and one for Implementation. Final class ready to produce system object (object which describe system we model) when it derived from proper implementation of interfaces exposed by our system.


Interfaces diagonal is redundant, it just a shadow of Implementation, has no properties and do nothing.
Implementation can be reused along Impl diagonal. Suppose we have 3 class diagonal and each class add one unique property. Model that can be build from it is power set of properties:

P {a, b, c} = {{}, {a}, {b}, {c}, {a,b}, {b,c}, {a,c}, {a,b,c}}

If no overwriting on parameters assumed, the only interesting model is complete diagonal - {a,b,c}.





C++ diamond inheritance model.


C++ employs virtual derivation schema when subtypes keeps parental properties or override them. This model can scale in any direction and is famous for its tendency for over-design.

Diamond models are domain specific and used mostly by frameworks.  No reuse achieved ever.

Previous (Diagonal) model scaling linearly from number of properties. But on diamond we can do permutations, so number of domain model described is factorial. 




Mixins model (includes: good C++, most of C#, Scala, all but bad JavaScript).

It's theoretical thus very orthogonal.

Prototype class
^
Component class (+) trait* 


Mixins is about sharing behavior between objects. There are anchors embedded into `Prototype class on which all traits agreed and synchronize. Anchor can be any kind ranging from enumeration to type constructors.   

This model is very powerful and can "bless" any class with additional behavior.

Comparative to previous discussed models  this one is real Functor. It works on constant property set (anchors). It creates new domain model by mixing traits.