Aldonline

Decomposing a Regular Transformation Matrix in AS3

If you ever find yourself building a GIS interface, mapping app or any kind of visual system that requires some sort of zooming in and out of complex nested hierarchies of visual objects, chances are you will need to use Matrix Transformations at some point.

Enter the Matrix


Matrix Transformations lie at the heart of display rendering because they provide a simple and efficient way to handle geometric calculations and other hierarchical transformations. In the case of Flash they are built into the native framework and surfaced through the Matrix class.

When you rotate, scale and translate an object, these operations affect its associated transformation matrix, which is then used for drawing and geometric operations. If this object is nested within another display object, then its associated matrix is combined ( through matrix concatenation ) with the matrices of all its ancestors. The resulting matrix is adequately dubbed the display object's concatenated matrix, and you can access it like so:

displayObjectA.transform.concatenatedMatrix;


Exit the Matrix

Manipulating Matrices and using them to perform complex calculations is quite useful, and we do it everyday. But, what if we need to go backwards?
What if, given a transformation Matrix we need to figure out exactly what amount of rotation, scaling and translation operations would produce it?

At first hand it seems like a trivial task. In fact, I have seen some people wondering why the Matrix class itself is "missing" the obvious properties: Matrix.rotation, Matrix.x, Matrix.y, etc...

Unfortunately, this is not trivial at all. Just a little experimentation will demonstrate the contrary.
The problem lies in the fact that operations are interdependent. Especially when introducing rotation steps.
If you translate and then rotate, and then translate again you end up with an unclear mapping between each individual step and the final result. In fact, you even need to take the order of the transformations into account. Different sequences can produce different results.

Go ahead and try to figure it out if you don't believe me. Fact is that there is no simple way to go from a Matrix to rotation, translation and scale.
But who needs simple anyway...

Eigenspace and Singular Value Decomposition


Luckily for us this problem is not new. We can resort to a set of concepts with fancy names: Singular Value Decomposition, Eigenvalue, eigenvector and eigenspace. In short, they provide us with a framework, theory and some alternative algorithms that allow us decompose a transformation Matrix into its constituent components and derive a minimum sequence of operations to reproduce it.
After a couple of hours trying different approaches to implement the math and and digging up old codebases ( of which Dojo was the most useful ) in search for ideas, I decided to save everyone else the pain of going down the same road and created a simple utility class that does all the work...

var c1:Panel = new Panel( );
var c2:Panel = new Panel( );

// nest the components
addChild( c1 ); // c1 is below root
c1.addChild( c2 ); // c2 is below c1

// apply some manual transformations to outer component
c1.x = 10;
c1.rotation = 5;
c1.scaleX = 2;

// apply some manual transformations to inner component
c2.x = 20;
c2.rotation = 10;
c2.scaleY = 0.3;

// access the resulting inner matrix
var m:Matrix = c2.transform.concatenatedMatrix;
// and decompose it ( THIS IS THE INTERESTING PART )
var svd:SingularValueDecomposition
    = SingularValueDecomposition.decompose( m ); 

// we can now access the individual transformation bits
trace( svd.dx ); // x translation with respect to root
trace( svd.dy ); // y translation with respect to root
// etc...
// or apply the transformation to a new display object

var c3:Panel = new Panel( );
addChild( c3 ); // c3 is below root

svd.apply( c3.transform.matrix );
// c3 now looks just like c2
//( scale, rotation and translation )
// but it is not nested


Notes

Now, there are a couple of important things to keep in mind.
The first one is that the apply() method follows some rules when applying the results. You can also apply each operation by hand. But remember to follow this sequence if you want to achieve the same results as the original matrix:

public function apply( matrix:Matrix ):void
{
    matrix.translate( dx, dy );
    matrix.rotate( angle2 );
    matrix.scale( sx, sy );
    matrix.rotate( angle1 );
}

Just think of this sequence as the quickest, generalized way to achieve all complex linear mappings (which include non-proportional scaling and rotation). And that is why you often end up two angles instead of just one.

Now, the second thing you should not forget is that this is an approximation.
Depending on the magnitude of scale, for example, you might see the fidelity of the results drop. This is due in part to the limited numeric representation internals: Numbers in flash can only hold so much info ( this is common to all platforms ). And since calculations operate on these numbers, and many calculations are required in each pass, some precision is lost along the way.

So, keeping the latter in mind, if you are planning on using this to achieve something like a really deep zoom-in effect (a la google earth, diving from outer space into your home... which is similar to what I was doing) I would recommend using an exponential approximation strategy, where you recalculate the trajectory whenever you travel 50% of what's left, for example, so you can gradually eat out the error.

Get the Code

The code for this class and its supporting classes are available from the trunk of the BigFlexLib project. There are no public releases yet for that project, but you are welcome to grab a copy and do as you wish.

See source here.

Flash VS Silverlight at the Olympics

So, the news about the fight are going mainstream.

Olympics set the stage for Web tech fight

SEATTLE--As the world's best athletes competed in Beijing, the summer Olympic games set the stage for a battle between Microsoft and Adobe over the Internet's next big competition.


( Nice simplification. Usual caveats and distortions apply. )

I'm glad to see more and more sources realizing that the real question is not whether RIAs are taking over the web or not, but which RIA platform is getting there first.

Now, don't get me wrong. I am all in favor of Standards. I have built RDF and w3c standards into every piece of software for the last 5 years. This fight between Adobe and M$ is in a lower level: transport, tools, network and technology. Standards will penetrate, eventually ( in fact, the web architecture IS the trojan horse of standards. It is just a matter of time ).

This only means that Semantic agents will look much better than originally planned.

The Nesting Enterprise Dolls Leave Many Orphans Behind

This is probably a *very* boring post. Cold, grey sunday style. Consider yourself warned.

I was toying around with an old system built on Apache Tuscany today, specifically trying to publish some apps as Linked Data ( SDO to Linked Data is probably an interesting gap ).
I ran into a simple harmonization problem and I remembered I had seen something conceptually similar solved in a totally different context, so I decided to go take a look.
Where?
Uff, it took me a while to remember, but finally I did. It was in E.6: E.piphany's core platform, which is/was the piece of software behind their CRM back in the days.

For those of you who don't know, E.piphany was an enterprise grade software vendor for the Sales and Marketing et Al. market. The product I got to know well from a tech perspective was their mammoth sized CRM and the platform beneath it. It was a tightly integrated MDA style platform with an IDE, a Data Bus, BPM, UI framework, Web Service/RDBMS Connector stack, etc. Pretty much everything you need to model the system, build it, plug it in, run it, tune it and make the tires hit the road... plus built-in analytics and a complete set of lifecycle management tools. Ah, and Crystal Reports could be thrown in as well.
Of course this sounds better than it actually is. Vertically integrated platforms like this sometimes feel like Titanic down a level 4 river. You need to do a lot of planning before grabbing the wheel, and steering is damn slow.
I prefer loose architectures, modular solutions and organic development ( REST, Linked Data, RIAs ).

If E.piphany doesn't ring a bell, then let me tell you that Open Source CRM rockstar SugarCRM was actually founded by ex E.piphany employees. If you look close enough you'll notice some heritage traces, mainly in the general architecture and the UI. Although things have evolved over time.

( Sleepy? I told you this post was boring ;)

Ok, I'll cut to the point.

So I set off to look for technical docs on E.6, to see if I could refresh my memory and find that piece of evidence that I was looking for and, if I got lucky, perhaps even get my hands on some evaluation license of the new version of the platform or its successor(s).

The thing is that E.piphany no longer exists. It was acquired by SSA Gloal Technologies in 2005. And SSA was, in turn, acquired by Infor in 2006.

Oops.
That's a lot of layers to dig through.

So I set off and started systematically Google-ing for terms I remembered, like BIO ( Business Information Object ) and others. But results were pretty scarce and I hit many dead ends.

E.piphany was a big company, with a big public image, lots of published documents, binary distributions, communities, etc. And suddenly I found myself doing some serious archaeology to find vague traces of it. In this Googly age we're living in, I found this to be quite a surprise... Where is the Brand? Where is the Culture? Where is that big orange E? Where is IT?

And then it hit me: E.piphany is... GONE.
Everything that made it what I perceived it to be no longer exists.

Paradoxically, however, the most important parts supposedly still DO exist, because the whole point is that the core ( know how, team and technology ) was bought and put to work for someone else.
It is the rest that has been discarded: cultural aspects, branding, PR, communication channels, etc. The infrastructure. The soft side. Apparently that amounts for a lot. Enough to make me feel totally disconnected and lost.

Like a set of Nesting Russian Dolls, you can only see the one on the outside. Whatever is inside becomes invisible.



I might be just saying some obvious stuff here, but my observation, and the real point of this post, is that in today's extremely competitive technological industry, some acquisitions cannot afford to destroy brands like this. They are loosing a valuable asset. I think they should tap a hole in the the dolls so you could see through.
Leave a trail so stranded guys like me can eventually find the way back home.

I know another story which is much closer to my heart, but in this case they pulled it off with remarkable ability: They embraced the culture, the vision and the community. They "merged".
Even then, however, I still miss one lovely doll from time to time ;)



Closing Thought:

The Nesting Enterprise Dolls Leave Many Orphans Behind :(

The Shirts are Coming....

Apologies for those of you who have been asking about the Global Mind T-Shirts.
We have hit one small issue that has been holding me back.
Since I am processing the images using the Flash player in a *browser* ( over a hundred images downloaded via HTTP, desaturated, scaled and overlayed ) I have been struggling with memory and performance.
It has been quite an adventure, and frankly I am impressed on how well the Flash player can handle all this work.
The problem is that I also need to generate a high-res image ( 3000x3000px ) and scaling has to be interpolated or somehow smoothed. So, you can imagine how incredibly heavy this thing really gets ( ahh... and I forgot the fact that we are also processing over 500 RDF-XML documents and asserting 10000+ triples. In memory ).

So, I am almost there, squeezing performance bit by bit.

Hmm... I wonder if Pixel Bender would help me a bit.

Now, of course I could pull this off quickly just by splitting the workflow and using an external tool. Photoshop batch, for example. But... why make things easy when they can be complicated?

Having said that, my guess is that I will need to use Photoshop at some point to smooth/color correct the images before sending them to the Print anyway.

Wow. And I thought I was a bit Hyperactive...

Meet the Global Mind! ( Girl or Boy? hmmm )

Ok, I wasn't supposed to talk about this till next week, but since Marshall K. is wondering about the gender of the SW. I guess it is my duty to share with you a picture we managed to take of it.
It looks pretty asexual to me, what do you think?:




This is how this thing came to be:

  1. We crawl the Semantic Web ( foaf-space, starting from TimBL ) and harvest profile images using semanticflash
  2. They are dynamically fed to an Adobe AIR based application that processes each image and creates the mosaic on the fly
The image you are looking at does not have actual profile images yet ( so don't try to recognize yourself, you'll waste your time ). It is just a snapshot of an older version that I took out in a hurry.

In case it isn't clear, the idea is to visually communicate that the SW is made up of all those that are contributing to it. In this case, it is made up of the seminal group of people behind the technology. In the future, everyone will be involved.

This is part of a much larger, stealth long-term project called the Univrz, part of which consists of experimenting with different metaphors on how the new web might look like.

The actual application is really quite dynamic, with some 3D effects ( Papervision ) and all that, but we are fine tuning it so we can port it to Flex ( the idea is to make it available over the web and that it runs without crashing your computer... not easy to do without some server side help ).

And it gets even better:
I am talking to Benji, who runs the Semantic Web community shop to make some nice T-Shirts and Mousepads, so stay tuned.

( I want the mousepad myself, that's why I had the team build this ).

Please send me an email with the word "TGM" in the subject to aldo.bucchi AT univrz.com and I will let you know as soon as the goods are available on the shop.

Alternatively come check this blog around friday next week.

( ...Mr Kirkpatrick, couldn't you wait just one more week? )

For those who don't know, the semantic web logo is embedded in the eye:


Hans Rosling and Marc Benioff May Have Something to say About Globalization

Some time ago I commented on Hans Rosling's legendary presentation at TED and the impression it made on me.

Well, today I ran into this video of him talking about Chile ( I am Chilean BTW ) and comparing it to the US and Cuba. For us Chileans its no big news that our homeland is already a developed country in some areas, but that it has a huge social gap and low social mobility. This means wealth is unequally distributed and world class education is only accessible to an elite, perpetuating the situation.



Having lived in other countries ( like the US ) with very poor international awareness, I really appreciate Hans for the insight he is providing into the data: You need to look at the details, each case is different.

Sticking to one single criteria to differentiate developing from developed, or potentially competitive from non-competitive countries is a dangerous oversimplification.

Globalization is going to put seminal groups of people all over the world at the same "knowledge" level very soon, and everyone will be forced to leverage their own strengths to survive. This requires insight into the nature of each culture and its history, as well as a critical and objective reassessment of the situation.

I know a couple of tasks that I wouldn't ask a fellow Chilean citizen to work on, but I also know other tasks I wouldn't delegate to a Stanford graduate. They probably both know the same ( today knowledge is free and accessible ), but they have a different civic and emotional baggage.

Of course this is no new idea, for years companies been mixing and matching elements from the top 10 developed countries.
But with the advent of the Web and the Knowledge Economy, the possible sources are expanding dramatically. In fact, innovation will start anywhere.

What do you think will happen in the next decade regarding information access?
How do you think this will impact the distribution of knowledge around the world?

Everyone will have the power to know anything, eventually ( within obvious limits ).

But that is not sufficient by itself. We also need a way to empower small groups to actually create something out of their knowledge without millions of funding.
How?

Perhaps these words by Marc Benioff might give you a hint:


Web 3.0: Anyone Can Innovate
Web 3.0 changes all of this by completely disrupting the technology and economics of the traditional software industry. The new rallying cry of Web 3.0 is that anyone can innovate, anywhere. Code is written, collaborated on, debugged, tested, deployed, and run in the cloud. When innovation is untethered from the time and capital constraints of infrastructure, it can truly flourish.

For businesses, Web 3.0 means that SaaS apps can be developed, deployed, and evolved far more quickly and cost-effectively than traditional software of the client-server era. The dramatic reset in economics should help CIOs finally break through the innovation backlog created by the cost and complexity of maintaining client-server apps.

For developers, Web 3.0 means that all they need to create their dream app is an idea, a browser, some Red Bull, and a few Hot Pockets. Because every developer around the world can access the same powerful cloud infrastructures, Web 3.0 is a force for global economic empowerment.


( ... go to the original article )

Sites for Bookmarking Images? No more fragmentation please...

Who needs 3 Cool Sites to Bookmark Your Favorite Images on the Web??
Why not just use delicious, or stumbleupon, or you browser?
Or any of the 200 accounts that a poor web2.0 user already has to manage?

Ahh, because they offer some extra features. Most importantly, social stuff ( sharing, tagging ) and a specialized user interface.

The social features, however, are available in almost any bookmarking service. It's the UI that makes these sites better for tagging images.

Actually, this is true for many different things. When bookmarking places ( geo tagging ) becomes a regular practice ( just around the corner with the new iPhone ), we will see new sites that will offer a better UI for bookmarking / sharing places.

And any combinations of sharable items might also create the need for specialized interfaces: images + places, places + websites, places + music + notes + events, etc.

So, you can see where this is going. We will have a million sites with weird names, that are actually doing the very same thing, only wrapped differently.

But people can only handle so much identity / access fragmentation and will eventually move to a unifying concept, or platform. The Semantic Web can help here, but only as a unifying data model and integration framework. We're still missing some pieces. The last mile.

My 2 cents is that generic knowledge management platforms ( like Twine ) will need to grow some sort of integrated "mashup editors" to cope with the UI specialization issue.

Eventually, we will start seeing the web through these agents, and the integration will happen on the back, not by using our keyboards and password managers.

Mashups people. You will create and share your disposable UIs in a minute, they will be decoupled from data and identity, powered by your knowledge agents ( like Twine ).

Will they teach some sort of visual programming in school someday?

Personal Web Agent Programming 101

Linked Data Metadata. Introspecting the Semantic Web

One of the early stages of the Semantic Web is finally materializing in the form of Linked Data.

In fact, there is a huge, free database out there:



10 Billion pieces of data, some say... and growing ( sounds big, huh? ).
So, why is almost nobody using it?

Well, it is relatively new in the first place, and there is some technological learning curve to overcome ( BTW: if you don't know anything about RDF then this post won't make sense either ).
Other than that, there are a couple of workflow issues that make integrating this data into your applications a bit cumbersome.

One of the key value propositions behind the Semantic Web is "Unexpected Data Reuse". However, a requirement for reuse is that the published data must be discoverable by someone other than the publisher.

Most of us working on closed enterprise Semantic Web deployments have been building handcrafted introspection/directory architectures to solve this issue and make distributed data discoverable, in controlled environments, for quite some time. But now that this is going big ( internet big ), we need some sort of consensus and a large scale solution.

The requirements are simple:

  • Find the Data
  • Understand the Data ( structure and contents )
The good thing is that there is already a solution in sight.
We will start from the back:

Understand the Data

It is not always necessary to understand the data. It depends on what you need to do with it.

If all you need to do is store it, then you don't need to understand it at all. Just find it, GET it, end of the story.
However, if you need to do some processing of the data, you need to know at least its encoding, syntax, formatting, etc.
If you want to use this data to create a magazine, then you need to understand its contents.
If you want to feed this data to an application, then you need to understand it in a deeper, more structured way.
etc. etc...

RDF already provides a common encoding, syntax and structure ( which is RDF itself ) so, in a way, Semantic Web data can already be understood at a structural level in an ad-hoc way. On top of this, there is an Ontology level which adds even more understanding, on a semantic level ( hello, "semantic" web ).

However, there are some bits of data "about the data" that might be useful when discovering and integrating that are not currently inherent to RDF. For example: the size of the dataset, its owner, the copyright license of the data, alternative access mechanisms ( archive, query endpoint, etc ).

This could be called Linked Data MetaData. Think of it as the attributes of a file in the file system.

Luckily, this issue has ( apparently ) finally crystallized into the voiD vocabulary, and it seems to be gaining mainstream acceptance. The community is considering most of the bits of data you would deem useful when dealing with a new, published dataset.

As a side note, one of these bits that is specific to SPARQL endpoints and that I predict will be immensely valuable in the near future is federated querying metadata. This refers to some simple statistics about the dataset that are necessary to come up with an optimal federated querying strategy.

Autonomous and federated ( aka distributed ) querying is probably one of the next big things on the web. Mark my words.

Wrapping up, RDF is already self explanatory, and once some version of voiD becomes deployed across the Linked Data Cloud, we will be able to understand even more about the datasets we find. Which takes us back to the first problem...

Find the Data

The data is out there: Linked DataSets and SPARQL endpoints are popping up everywhere.

But, currently, there is no central way to discover them.
Sure, you can Google for "RDF music" and you will hit MusicBrainz, or you can start from DBPedia and follow links outwards to other datasets. You can also review the LOD cloud diagram and try to figure out an entry point to the datasets on your own.

It is evident that we need a smarter way to discover data.


Enter Sindice:



Sindice is making a disruptive contribution by indexing the Semantic Web and providing a set of APIs and resources for developers to build upon. Among other things, it provides a way to:
  • bridge the gap between keyword search and RDF
  • find datasources mentioning a given IRI
  • work with IFPs
  • ( ...checkout the website for more information )
Apparently, they will be also soon provide some introspection functionality fed from voiD metadata and from massive mining of their index.

For us humans, Sindice also provides a nice page with links to popular published datasets.

Near Future

The sharp minded might be feeling a bit uneasy about Sindice.
After all, isn't centralization against one of the core principles of the web?

The answer is both yes and no.
It depends on the timeframe I guess...

First, in the short term no one can argue against Sindice's immediate value proposition and contribution to Semantic Web adoption.

In the long term, I hope to see an architectural pattern emerge from this centralized Semantic Index ( Sindice ) that will eventually be copied and distributed, giving birth to some early form of introspection DNS-like layer. This layer should also be implemented in RDF.

While similar subjects ( directory and discovery ) have been studied for Web Services and Semantic Web Services, the overall problem is simpler in this case ( we are only reading data ). Therefore, things should happen in a smaller timeframe and hopefully propagate swiftly throughout the web.

When this new layer crystallizes, we should be able to point our agents and IDEs ( for query building, for example ) to these Sindice-like Linked Data Cloud Introspection Agents.

Is that a good name? Introspection?
Or just search...