I had the opportunity to work on some non-music related projects recently, including a project that involves analyzing mutual funds.
Investing money in the stock market is pretty easy. However, making good investments takes considerably more skill, especially in the current economy.
Mutual funds are one way of managing the complexity of the market. Rather than choosing a number of stocks, and then determining how many shares to buy of each, an individual can purchase shares of a mutual fund. While mutual funds sell shares like other stocks, they don’t actually produce goods and services like a conventional publicly traded company would. Rather, they take the money they receive from the sales of their own shares, and then go and buy other stock with it. Typically mutual funds focus on investments that have certain features, such as stocks that have a large market capital, or stocks that are poised to grow significantly in the next few years. In this sense, they seek to emulate certain basic investment styles that many investors want to follow. In some sense, mutual funds can be thought of as “savvy” investors. Understanding relationships that they share could go a long way to understanding how the investment community understands opportunities in the stock market.
The availability of mutual funds still presents a good deal of complexity for the average investor. There could potentially be dozens of funds that might match their chosen needs. Are those funds fairly equivalent, or do the funds differ wildly in what they hold? There is still a lot of research to be done in most cases.
MorningStar has a rough approximation of how basic stocks relate. Their Market Barometer indicates gains and losses over different market sectors. These sectors include companies that have small to large capital, as well as companies that are geared for growth (risky) or value (conservative). ( I seem to have picked a bad day to get my example from, as all sectors are down. Typically, there would be a bit of green somewhere).
I thought it might be interesting to try and see how the mutual funds related to one another from the bottom up. That is, how well do they follow the market barometer grid according to what they actually invest in? Are the small cap funds always more related to mid caps than large caps? Are growth funds always more related to core than value funds?
It’s possible to see what the mutual funds hold on a quarterly basis. All funds are required to submit their activity to the SEC, which offers these records publicly through EDGAR (Electronic Data-Gathering, Analysis, and Retrieval system). It’s possible to extract information from EDGAR, and combine it with historic stock pricing information to get a fairly good representation of what the portfolios own at regular periods.
Understanding how funds relate to one another can help to assess a few important issues of risk:
- There is strength in numbers : Typically, if a number of different funds follow a given portfolio allocation, then it’s a sign that a number of very savvy investors believe that this given configuration is a good bet.
- There may be opportunity in outliers : Simply following what everyone else is doing will never allow you to outperform your peers. If funds want to set themselves apart from their peers, they will need to change their allocation ratios, or add new stocks that their peers do not own.
- There is usually noise in outliers : In some cases, a given fund may behave very differently from their peers, but perhaps they are not distinguishing themselves from their peers in a useful way. For instance, they may advertise themselves as following a small cap growth investment style, but have a portfolio allocation more similar to a mid cap investment style. In other cases, they may not report their holdings correctly to the government, leading to errors in how their portfolio is related. In both cases, their behavior could be a negative indicator for opportunity. In the first case, they’re not strongly related to their peers, but they are strongly related to other irrelevant funds. In the second case, they are not organized enough to submit properly formatted documentation to the government, indicating that they may lack in some administrative areas.
The core component of my visualization technique is an advanced multidimensional scaling method, and it’s a bit too complicated to go into the details here. However, it’s essentially possible to turn thousands of portfolios into points in two dimensional space, where the portfolios are arranged to preserve their similarity as a geometric distance (similar portfolios are close together, dissimilar portfolios are far apart).
The interesting thing about the resulting scaled representation is how similar it is to the market barometer layout. In particular, the large cap funds (orange to red) arrange themselves in order from growth, to core, to value. The grid itself is slightly rotated and reflected, but this is an irrelevant artifact of the dimensionality reduction technique (rotating/reflecting the points does not affect their distances in two dimensional space). The mid caps and small caps are separated somewhat from each other, but not as strongly. They also do not arrange themselves clearly from growth to value like the large caps. This is to be expected since there is far more diversity of stock options for these fund styles to choose from (there are more small cap stocks than large cap stocks).
I can select individual funds from a small region of the plot, and verify that their holdings are more or less the same. For instance, the above plot shows three funds taken from the same location on the plot. While they reflect three different styles (an S&P 500 index fund, a large cap value, and a large cap core fund), they have very similar portfolios. The bar plots indicate the value of their individual stocks, and the top stocks are labeled with their stock symbols.
The final element I added was to animate this visualization over time to reflect how the funds changed their relationships with others by buying and selling stocks. It’s interesting to see one or two outliers that identify a hot stock suddenly turn into a small cluster of funds. It’s also interesting to see other outliers hover and move over different regions of the visualization as they emulate a different style. There are also plenty of funds that are outliers because they are missing information, or had improperly formatted data. I was able to remove most of these using the visualization itself to identify them.
You can see the visualization in the movie at the beginning, but it is much clearer if you see it in high definition here.
The upper left legend shows the colors used for the fund type acronyms. LCC = Large Cap Core, MCG = Mid Cap Growth, SCV = Small Cap Value, etc. Also, SP5I is an S&P 500 index fund.
The upper right legend shows the coloring scale for the “halo” (the colored border of each fund). The green to red scale is an indication of the change in value from the previous month.
The bottom legend is an indication of the total portfolio value in dollars, scaled logarithmically, and expressed as the plot point circle area.
I finished up a rough version of my t-SNE package for R. If you haven’t heard of t-SNE, and are into dimensionality reduction and/or visualization. You should check it out. The “tsne” package for R is available on CRAN.
t-SNE is a non-metric multidimensional scaler. It creates an embedding by a gradient descent process, much like isoMDS, and a number of other approaches. However, it does some interesting things to the high dimensional data before it proceeds.
First, it translates the matrix of raw numeric values into probabilities. The probabilities are a guess as to the likelihood that another given datapoint is a neighbor. The number of neighbors that are considered is parametrized by a “perplexity” argument, which is the optimal number of neighbors for a given datapoint. A gradient descent attempts to preserve aspects of entropy present in the probability distributions.
The technique handles large variances fairly well across the values in the dimensions or the datapoints themselves, and is fairly quick in the applications that van der Maaten provides. However, my R version is unfortunately much slower.
In the meantime, I’ve been testing it out against various datasets. It does seem to provide a more natural clustering in many cases, such as the classic “iris” flower measurement dataset available in R. I’ve made a simple animation of how the gradient descent process sorts out the clusters and gradually arranges them to minimize its error function:
This produces a better cluster than a comparable PCA technique, based on simple covariance:
I’ve tried it on some other datasets, and it works even better with larger sets of more complex data. I’ll update this post with a link to the package once it goes through the verification process at CRAN.
I’ve switched the host for the blog over to scwn.wordpress.com. The scwn.net domain points here as well. Some minor things like images and other media may not show up correctly, and I’ll try to add them back in as I go.
The 2009 Music Industry Panel has just kicked off, and since Paul Lamere is on the panel, he won’t be able to blog or twitter about the events like he has so far… so I figured maybe I should do a brief write up instead.
The panel is comprised of:
- Paul Lamere – The Echo Nest
- Tom Butcher – Microsoft
- Peter (?) – Gracenote
- Oscar Celma – Barcelona Music and Audio Technologies
- Norman Casagrande – last.fm
- Keiichiro Hoashi – KDDI R&D Laboratories
- Kunio Kashino – NTT Laboratories
- Malcolm Slaney – Yahoo
Paul is acting as an ad hoc moderator, and started off asking: What are the tough problems that people in the industry are facing right now?
Tom : We’re actively trying to grow our music market, and come up with recommendation engines that operate at large scales. Overcoming cold start problems, and evaluating user assessments in a general way is difficult.
Peter : We have special challenges as a b2b company, since we have a lot of external constraints. We have to cater to the needs of the customers, and accommodate what they want to pay for.
Oscar : In our case, we have a lot of practical problems: metadata cleaning, etc. I’m interested in semantic music descriptions, and music discovery in the long tail (less popular music). We have clients & music from smaller markets like Turkey.
Norman : Metadata is one of our biggest problems as well. We try to leverage our users to correct some of the metadata, but you cannot rely on this. We also face scalability data, in the face of real-time data needs for our users. We have to be able to allow users to build a profile, and then provide recommendations immediately. We also focus on understanding user “types” and general listening behaviors. We also face the limitations that the record labels impose on us. We are provided a pool of tracks (several thousand), and then we apply some user listening behavior metrics on this data. Then, once we have a collection of candidate tracks (from the label), and the label restricts which songs can be played where. We end up filtering a lot of the music out because of the limitations of how we can match against their catalog. Evaluation is also really important, but AB test can be tricky… it is easy to become too biased, and it’s important to design user tests correctly.
Peter : There is a lot of weird grey areas here, you can’t put cover art in embedded devices for instance. The music publishers have the right to lyrics, which we have to explicitly clear with the publisher, and sometimes they are too slow, or not willing to clear rights in certain cases.
Keiichiro : We do our own music distribution service, and we are trying to compete with the “giants” such as iTunes, to increase sales and preserve the rights of the artists. On the research side, we are trying to convince the music people to know that MIR is a “good thing”. Our service has a lot of MIR functionality, but most of the services are based on conventional search. My research is on more discovery search, but many business people are not convinced that you can present unknown songs to the users as recommendations.
Kunio : In Japanese language, there is many different ways of writing the titles of different songs. So, we can’t always identify music based on simple labels. We also have difficulties with dealing with all the different variations in recordings, and deciding if the recordings are actually the same. We are actively researching this.
Malcolm : We don’t have a big problem scaling, but we have a problem deciding on how to provide the right content to the right user. On average we get 2.2 words per query… so we have only 2 words to figure out what the user wants… Madonna the music star? Madonna the painting? etc.
Paul : If you look at mirex as where people are spending their time, and we see classification of genre, artist, etc… and then we look at the companies actually doing something with music (apple & itunes, guitar hero, google music, pandora), they don’t tend to use these techniques. For the panel, the question is… why do you think that is? Has the industry not caught up, or is research in the wrong area.
Malcolm : None of my people are begging for these technologies
Tom : We are not doing content analysis yet, it’s hard to do currently with our dataset. When we launched the collaborative filtering algorithm, we had a lot of good data for that. The product people don’t come to us and ask about content analysis technology, they want to know how to launch a service in Germany. I disagree with the fact that people aren’t using MIR techniques in the industry (SongSmith). It takes someone that knows about the research to able to see the solutions and opportunity for MIR techniques.
Keiichiro : There were two concerns for using MIR techniques: The cost of performing content analysis, and the fact that we didn’t have the rights for songs. We had to ask record labels and the artists if they agreed to allow us to analyze/use their content.
Oscar : I don’t think the mirex tasks need to be useful for the industry, maybe in 10 years it will prove useful or will be mature.
Peter : In all of these situations, industry comes down to a question of profitability. The customers will only buy if they’re convinced of profitability. Apple made decisions on recommendation technology, and the decision was cost based on the incremental sales it would provide. Also, number one, everyone is working on new things. The first is that different combined techniques are being considered over individual approaches. The second is to ensure that the solutions work very well globally. It can’t just work for certain types of music. You can’t exclude yourself from certain international markets.
Tom : You mentioned Harmonix (Guitar Hero). The technology in these games has changed how we consume music. Not only are they fun, but the music in these games is monetized and consumed in new ways. It’s more interactive, so you can charge more for it. That’s a good way to make money.
Paul : A nice thing about the industry is that we have more direct contact with users. There’s not as much research around taking advantage of user data. What kinds of data do you have that you would want to share with researchers.
Malcolm : I think AOL proved that giving out user data is a firable offense. We give out CF data that has been thoroughly scrubbed. We have a rating database available of 700 million rating data.
Norman : We have an API for a lot of data. But it’s also true that I’m not a user of this API. Quite often, we’re not offering something that’s not useful to researchers. For instance, we only return the last 200 tracks for a user. It used to be a practical limitation, but not necessarily anymore. We are willing to negotiate directly with researchers if they require more information.
Tom : What Netflix did with their rating data really opened the door. The contest really drove innovation in the field. The other thing I’d like to mention is that Microsoft has a great internship program with the research labs. In that setting you have access to all the data you want.
Paul : We’ll open up from questions from the audience
Question : What kinds of skills are you looking for?
Malcolm : I interview a lot of people. From the research jobs, I’m looking for someone who can teach me something new. For the engineering jobs, I want to see that you can build something. show your passion. Show something nobody has done before.
Oscar : We are looking for music lovers. We use typical tools like C++ , but we mainly need people with open minds who are willing to try new things
Keiichiro : We need people that can think of things that corporate people don’t. However, we only hire people who speak Japanese… so that might disinterest most here (it’s a small problem).
Norman : When we are hiring at last.fm, we are looking for people that have previous experience with small toy datasets. Something that shows passion, etc. Our team is very small, so the person we want is someone that can do a lot of coding, research, and development. Also ping-pong skills are important.
Peter : The people that work in Markus’s team have to be Jacks of all trades. We need people that can build complete systems. People with motivation and drive.
Paul : When I was at Sun Microsystems, we were in a feud with Microsoft. It never ended to surprise me how many resumes I got in Microsoft Word. Paying attention to the company you are applying to makes a lot of difference 🙂 The people that we hire we tend to invite first. What that translates to is that we are looking for people already doing stuff in their spare time, and making it public somehow. That is a good indicator that they have a lot of passion.
Kunio : We are looking for people with a vision of technology and services. The successful students are expected to have their own core technologies, and an ability to express their visions.
Question : I’d like to ask you about query by humming technologies, as well as illegal downloading issues.
Tom : I haven’t seen any commercial applications of this (query by humming). It’s not in the works for our mobile services.
Keiichiro : We actually have a query by humming service. If anyone can beat our service, we would be interested.
Paul : In the west, there doesn’t seem to be much interest in query by humming. Is there more interest in Japan?
Keiichiro : I’m not sure how much it is actually being used, but we like to try a number of approaches.
Peter : We don’t have a working example, but it is something we’re interested in. As the technology improves, we will start considering it more strongly.
Question : What are the differences between the use of symbolic data and content data?
Malcolm : I think symbolic data means text to us.
Paul : We use all of the data we can.
Norman : One of the biggest challenge we face is how to combine those different sources. It’s hard to figure out what is best. Which one do you trust? What are the priorities.
Question : Are you scared of Google’s music service?
Malcolm : We already compete with them.
Oscar : We are looking forward to crawling their data.
Norman : We are interested in stealing their engineers.
Tom : I think Apple is really the big player in music. Google music is based in iLike, which was started in Seattle. If you’re feeling entrepenurial, that’s a great example of how to make some money.
Paul : I like what Google is doing. If you search for a banner there, there’s music links in there, which lets you listen once or twice to a given track. Some of the PR said that people went to Google first for “discovery”. I don’t think that anyone will actually find anything new given how Google works.
Question : What are you all excited about these days?
Tom : From my experience, what we are looking for is ways to augment the way they consume music. We don’t own the content, we don’t generate the content. Any revenue we generate is around the services we build around consumption.
Keiichiro : I agree with Tom. When the music service started at my company, nobody knew how people were really going to use music (as an alarm clock, or as an e-mail notification, etc.). We are looking for new experiences and ideas (like our radio example in the demo session).
Paul : One thing I’m really excited about is services like Spotify’s new API. As Spotify spreads its domains, other companies might do so as well. I think labels might realize that music can flow more freely, and they might relax their copyright control. That will lead to a lot of new ways of interacting and consuming music.
Question : I’m interested in music creation/composition. What can you say about supporting tools for music creation? What do you think is the next generation of MIR technologies useful for composition?
Tom : We don’t represent the manufacturers of music creation software. We had this application called SongSmith. You sing unaccompanied into a microphone, and the application generates the accompaniment. That idea was generated by one of the developers.
Oscar : The Yamaha Vocaloid technology came out of MTG at UPF, which we spun off of.
Question : How good does a technology need to be to make it as a commercial product? What is your criteria for quality?
Tom : That’s a good question. However, it doesn’t matter. The technology could be good or bad, but if it satisfies a consumer need, then it’s “good enough”.
Malcolm : At the high level I agree, but at the low level, we just look at traffic on our website.
Question : A lot of you to one extent or another have been following this conference for some time. What do you think is a highlight from this conference that has helped at your company?
Oscar : I’m interested in merging content with context, which has been presented here.
Malcolm : We’ve come a long way since using simple MFCC features… but it’s hard to point out a single example.
Norman : The MIR has been building up datasets and tools over time, so bits and pieces come from many places.
Paul : All of the work on tags has been very useful.
Question : Do you work with Non-western music? Do you have specific problems with that?
Norman : We are in that situation. We have problems with label changes due to languages. Those are issues we have to deal with.
Keiichiro : Most of the music we work with is Non-western. We had some cases when we worked with ringtones. Some are not music at all. That messes up the music features we were expecting. If metadata was present, we could solve that problem.
Norman : There is also an issue with bias. The majority of our users are in a few countries. When you compute similarities, you are biasing towards the cultures there.
Peter : We put a lot of effort in getting to a baseline performance of good coverage, and good support of local cultural sensibilities. We have over 2000 genres, and we try to use local experts for editorializing in certain contexts so that everything is consistent to the region. This also applies for mood data and classifier. We try to do our best to make sure the training data has information from all those places in the world. There is a long ways to go still. We spend a lot of time in how to map classes and categories to labels and terminologies appropriate for different markets.
Malcolm : Yahoo’s goal is to organize all the world’s information. Most of the revenue and users come from bigger countries, so it’s doubly hard to work elsewhere.
Paul : We talked about long term, grand challenges earlier. What are the big problems being faced in the next 5 years?
Malcolm : Maybe give us 10 years. What our product people are asking for is understanding content. We want to understand user content. We want to be able to identify music embedded in other content (music in videos). How do we match music in other media?
Norman : One of the big problems is finding the “little gems” in the long tail. Some of the popular music deserves to be less so. From a research perspective, coming up with a better qualitative measurement would be really cool. I don’t think there’s a generic value for quality, so it should be based on user interest. I don’t think the quality of the content based recommendation is there yet. I also like analysis of sub-song quality… identify smaller parts of songs that people really like.
Oscar : We are interested in understanding the user better. We’d really like to improve that.
Peter : I also have user profiling and personalization at the top of my list. There’s been some great advances, but it would be nice to have a solid personality profile for people, that would enable a better understanding of what people like, etc. Navigation and visualization is also needed, and there are a lot of unsolved problems in this domain, especially in mobile devices, cars, etc… not only to look cool, but to actually be usable.
Tom : I’d like to add to that. I’d like a way to bring things to customers, without them having to ask for it. The more the user has to ask for, the more chance there is for frustration. We think about recommendation experience as a single modality. It may not fit in all situations, but closing the gap on user expectations and needs is important.
Malcolm : In the real world, you don’t get much information from the user. We need more technologies that need less data to drive recommendations.
Question : How do you feel about interoperability of the different services?
Norman : I think it’s awesome. As long as our users are using it, and are aware of last.fm and provide new data, then it’s great.
Paul : There are some need for standards. Some standards are proposed, but businesses have been slow to adopt. I think there may be some interesting things come down the line there.
Tom : I’m very disappointed with my company’s stance on this issue. We are a closed model like Apple. I’d like our group to take a leadership role in this area.
Question : Do you know a technology that was good enough on its own without any marketing?
Paul : Shazam was an example like that
Malcolm : Most of Google’s stuff is like that
Paul and I recently finished up our tutorial on Using Visualizations for Music Discovery. The fast paced, 3 hour long presentation covered nearly every instance of music corpus visualization that we had found in the literature, or in casual “fan” based renditions. This included visualizations of artist connections generated from millions of playlists by high powered supercomputers, all the way down to personal hand-drawn sketches on the back of a notebook.
I’ve embedded the slides from the tutorial above. However, the format of the online slides do not include any of the video segments that we included, nor does it include the demo visualizations that we generated live on our laptops. While our demo code (as well as the rest of the tutorial materials) is available online, I also prepared some back up video versions of my demos, just in case something went horribly wrong during the presentation. I thought it would be a good idea to post these videos online for posterity.
The demo focused on exploring the “acoustic space” of music involving the lute and the folk guitar. Both instruments can be similar acoustically, but generally take part in culturally distinct genres. In the realm of audio-based genre classification, renaissance music that features the lute can get confused for folk music songs that feature the guitar. I wanted to focus on representative lute and folk music for the demo, and see if visualization helps to show the confusion that a classification might run into, and perhaps lead to ways of resolving it.
Rather than show all of the acoustic features at once, I chose to look at them one at a time. The first feature that I wanted to focus on was pitch information, or a general representation of what pitches, keys, etc. were present in the music. For this and all demo videos, the blue nodes corresponds to lute music, while the red nodes correspond to folk music. However, there are some “mislabeled” songs, as we will discover later:
The visualization shows how adjacent songs do in fact have similar pitched melodies, but can have very different styles or genre. This makes sense, since most popular genres use the same western arrangement of major and minor chords, etc. So, pitch information is essentially useless to separate the lute and folk music in our data.
The next acoustic data I looked at was timbre, which is a representation of the song’s tone or color. This type of data is typically the “best” information to use for genre classification, and I was curious to see how the timbre separated these genres:
Interestingly enough, the timbre space “almost” separates the music clearly. The softer tones of certain lute music are distinct from the rest of the folk music, which has harsher or brassier coloring. However, the lute has a variety of timbres that it is able to create, and once the lutist uses this harsher plucking action, it sounds more like conventional folk guitar music. At this point, it may be possible to make a “pretty good” separation using a hyperplane, but as indicated by the visualization, there would most likely be some significant error using timbre alone.
Luckily, our database of music included non-acoustic features, such as tags. The tags were simple terms or phrases that were applied to songs by human listeners. Using appropriate text retrieval techniques allows us to see how the music was separated according to how it was tagged:
Here we see the two styles of music are clearly separated. However, there is a problem… I had originally used the tag data to identify/label lute and folk music in the first place. I quickly found out that some of this music was mislabeled… i.e. classical lute music was labeled as folk, etc. So, even though the separation was clean, we now have exposed a new form of error.
The final approach that I tried was to mix the term and the timbre data in a “hybrid” representation:
This had the best result of any of the approaches that I had tried. It is no longer possible to separate the music by color (some of the red folk music is included among the blue lute cluster), I show that these red dots were actually mislabeled. In this fashion, the combined term + timbre features were able to “correct” each other, and a more valid representation of the genres were presented.
The code for this is all available on the website. The code is written in R, and requires the packages rgl, RGtk2, as well as a GUI interface package I wrote called sculpt3d.
Also, if you’ve made it this far, you should check out (the co-presenter) Paul’s blog. He’ll be live-blogging the rest of the conference as it continues through the week.
There’s typically a strong tendency to separate and process a single visual scene into two categories, called “figure” and “ground”. The ground category specifies some manner of background information. In a photograph, the “ground” may literally be the ground… for instance a grassy field stretching off for miles in every direction. Then there’s the “figure” category, specifying some sort of feature that is contained within the ground. In a photograph, the “figure” could easily be a person. (There’s a nice article on scholarpedia that gives a more authoritative description of these distinctions.)
In these cases, perception is a process of splitting the figure from the ground. In most cases, the figure is the most “important” feature to attend to in a visual scene. The ground component of a scene is typically comprised of low level, high frequency “textural” patterns (the dunes of a sandy beach, the rows of a cornfield, even the grid of a city street). It may be pleasant to look at, but typically not engaging without some form of figure present in the scene.
I was thinking about this with regards to fractals. Fractals are well known for their bizarre shapes and emergent patterns. They form patterns reminiscent of simple geometric shapes, but these shapes hide incredible complexity. I think that part of what makes them so visually compelling is that they consistently blur the line between ground and figure through the course of their animation (typically, a long zoom on one portion of the fractal image). A seemingly simple shape that we process as “figure” becomes a “field” itself under a constant zoom, and new fractal features leap out at us from the new level of detail. In this sense, a fractal is constantly “becoming”… there is always some new figure that will be generated from the field, and therefore, our brain has to struggle to keep up. Pair this sensation with the sudden realization that many of the fractal patterns reoccur (the main Mandelbrot image set itself will repeat at various levels of zoom), and it’s a constant visual cocktail of emergence, recognition, and momentum.
Anyways, it occurred to me that fractals haven’t been harnessed for any sort of visualization application. I wonder what aspects of fractals can be used to attract/focus attention, express relationships, or designate some form of measurement?
Paul and I have just launched a new blog:
http://visualizingmusic.com/
from the about page:
As the world of online music grows, tools for helping people find new and interesting music in these extremely large collections become increasingly important. In this blog we survey the state-of-the-art in visualization for music discovery in commercial and research systems.
Paul and I wanted to create an ongoing, up-to-date resource of all the different instances of musical “discovery” visualizations that we have come across. We’re defining this as broadly as we can, while avoiding musical signal visualizations, such as all the iTunes/winamp plugins. Essentially, music discovery visualizations should help you understand relationships between songs, and hopefully help someone identify new and relevant content that they had not heard or considered before.
There is an incredible variety of techniques used to express musical relationships with images, rather than words. There’s many people who believe the best way to communicate about music is not through words. As the famous saying goes: “Writing about music is like dancing about architecture”.
Be sure to check it out, and if you have any suggestions for new visualizations, or comments in general, don’t hesitate to contact either Paul or myself.
Update: This tool is now available on cran.
Researchers in academia and the industry consistently use visualizations to better understand their data. The standard two dimension scatter-plot is a staple of many exploratory data analyses. However, there are many cases where two dimensions are not enough. For instance in the plot below, I’ve set up a pairwise plot between three distributions (x, y, and z). The plots are duplicated on opposite sides of the diagonal (showing x vs. y …or… y vs. x). I find it extraordinarily difficult to perceive the three dimensional structure of this data solely with the series of two dimensional plots shown here.
Luckily, R has a package called rgl that lets me plot in three dimensions using OpenGL. It even allows for simple interactions with the mouse, like rotation and zooming.
This allows for a much better sense of the underlying data… but It’s still a bit limiting. Many times, there is “interesting structure” in the three dimensional representation that I would like to focus in on, or perhaps there’s just a lot of junk that I want to get rid of. Many times this structure is not isolated in one or two dimensions, but only becomes apparent through rotating and zooming the three dimensional display.
RGL has a useful method called “select3d” that involves selecting points in the plot, but it’s a bit tricky to use. Using select3d involves typing into the R console, then clicking and dragging on the plot. This produces output from the select3d function. However, the output of the function is… another function! It is then necessary to apply this produced function on the original data to determine which data points fall inside the selection range…. and then it’s completely up to you what you do at that point… do you filter them out? Crop them? Color them differently than the others? It becomes necessary to keep a further supply of methods at hand that can perform these routines on the output of select3d.
This is still all a bit too much to keep in my head for extended periods of time. It’s also a pain to constantly switch back and forth between the three dimensional plot environment and the console in order to pare down the data.
So, I’ve cooked up a little GUI tool bar that makes selecting, labeling, cropping, and deleting points in the RGL plot much simpler. It’s called “sculpt3d”, since it’s focused on altering and shaping the underlying data. In order to launch the toolbar, you first install a small library, and then make a method call: sculpt3d(x,y,z) where x, y, and z are the three dimensions you are interested in plotting.
I put together a small video that shows me playing around with it:
In the video, you can see how I choose a selection color (a mint green color that I thought would be noticeable against the bright rainbow colored points). Then I crop the data on this selection. This is a bit jarring because the crop function automatically zooms in on the selected points, and reverts them back to their original color. The delete function works in a similar way, and is a little easier to follow.
Once I have a smaller collection of datapoints, I might be interesting in labelling them (assuming I passed labels as an argument to sculpt3d). I can toggle the labels with a click of the “Label” button.
If I’m interested in saving the results of the pruning and cropping, I can access the currently selected points by calling the function sculpt3d.selected(), or the currently visible scene by sculpt3d.current(). This returns the same sort of logic vector that select3d does, so I can now use it to filter my dataset, save it under a new name, and come back to it later on. Furthermore… since it’s in R, and uses a cross-platform gui (GTK+ via RGtk2 and Glade), it’s possible to use this tool and the data on any platform I want to.
Currently, I only have this tool on a local webserver, and it’s still a bit rough. However, I thought I’d make it available. To try it out, enter the following in R’s console:
install.packages(rgl) install.packages(RGtk2) rep='http://ethos.informatics.indiana.edu/~jjdonald/r' install.packages(sculpt3d)
It’s currently source-only, so if you’re on windows, you’ll need to install RTools in order to compile everything. You’ll also need the GTK+ framework and Glade (which should get installed automatically with RGtk2).
However, once that is done, you can check the demo out by entering:
library(sculpt3d) demo(sculpt3d)
Thanks to Daniel Adler and Duncan Murdoch for rgl, and Michael Lawrence and Duncan Temple Lang for RGtk2. Between the four of them, they’ve produced a lot of great stuff for R.
I recently gave a presentation on haXe at the Strands offices in Corvallis and Seattle. I think it raised a few eyebrows, but probably ended up raising more questions than it answered. This was to be expected, as haXe is pretty unique in its ability to target multiple web-related platforms. The devil is in the details, and so I thought it was best to post everything here on the blog so that interested individuals can go over the specifics in a more involved way.
The demo that I showed was actually two parts: A simple “hello world” trace example, which I compile to swf, php, and javascript. Then, a more complex example that compiles some code to each of those targets, and then uses conditional compilation to build a web page that displays output from each target all at once (The PHP target references the javascript source, a javascript method references the swf file).
This, to me, is perhaps the most satisfying aspect of haXe: The ability to treat each target as a minor variation of an API, rather than as a set of totally unrelated platforms. Developing, targetting, and displaying results simultaneously is a strange feeling. All of a sudden, javascript, php, and swf are operating off of the same script, and you are totally in control.
You can download the source as an archive here, or browse the source here. Once you compile the Demo2 file, you should see something like this.
There’s a README in the source that’ll explain what’s going on.
If you’re totally new to haXe, you’ll want to download the compiler from the haxe.org download page. After installing, you can run the build1.hxml – build4.hxml examples by simply typing in:
haxe build1.hxml
For each of the build files. For the build.hxml file (non-numbered), you will need to edit it so that it points and outputs to a working web folder. If you use TextMate, you can download the textmate bundle for haXe, and use the editor to browse and build files.
Nicolas just added a new keyword to the haXe compiler called “using”. In short, this keyword lets you inject additional methods into instances of types.
“using” explained
The haXe “using” keyword is a special import command for classes. The command will import the given class normally. However, it has special behavior if the imported class has any static functions accepting an argument.
The “using” keyword takes any static functions defined in the class, and adds those static methods to the argument type, as if the argument type defined the method itself. In this sense, it behaves like a sort of implicit mixin, or trait. However, it’s also very different from these approaches, and to the best of my knowledge, nothing like this exists in any other language.
Explaining what’s happening in simple prose is actually a bit cumbersome, so here’s some example code. First, we’ll give a simple haXe example without “using”. The first class is a Demo class that contains the main function:
// in Demo.hx
class Demo{
public static function main(): Void
{
trace(BarStringClass.addBar('foo'));
}
}
The second is a class that contains additional methods:
//in BarStringClass.hx
class BarStringClass{
public static function addBar(str:String) :String
{
return str + ' bar';
}
}
If we compile the code using main() from Demo, we’ll get “foo bar”. We’re simply using the static class from BarStringClass to process the string and tracing it. Pretty simple.
Now let’s look at what we can do to Demo.hx with “using”. We’ll use the same BarStringClass from before, but we’ll change Demo slightly:
//in Demo.hx
using BarStringClass; // here's the 'using' keyword in use
class Demo{
public static function main(): Void
{
trace('foo'.addBar());
}
}
haXe now lets us access BarStringClass static functions as if they were functions of the String class. In this fashion, the standard base classes can be augmented with additional methods without the need to extend them explicitly.
Notice how in this example it wasn’t even necessary to specify the “str:String” argument required by addBar(). When the haXe compiler injects the methods from BarStringClass, it will automatically fill in the current instance as the first argument… turning BarStringClass.addBar(‘foo’) into ‘foo’.addBar(). All of the first argument Types of the “using” classes will be augmented in this way.
So in this case, BarStringClass had one static function (addBar), and the first argument type was String. Therefore, every String instance now gets the addBar() function. You could have many different Static functions in a “using” class, and each of the first argument classes will get injected in the same way.
“using” with types and/or classes
The other nice thing about this approach is that it works with the more broadly defined Types/typedefs, and not just Classes. So, my IterTools functions that work (mainly) with Iterable Types can be injected with “using”, and then you can do things like:
for ( i in [1,2,3,4].cycle(5) ){
trace(i);
}
…to iterate through the numbers 1 through 4 five times. “cycle()” and the rest of the functions that operate on Iterables would also then work with Lists, FastLists, Hashes, and so forth. When you add “using IterTools;” at the top of a class file, anything that can iterate can now “cycle()”.
Caveats
There are two significant caveats with “using”. The first is that the injected methods are not available through reflection. This means they’re not part of the “run-time” class instance. The second caveat is that it can be very easy to overload several instances of a single method name (from multiple “using” classes). Currently, haXe’s behavior is to only use the first matching static method from a “using” class.
“using” can be habit forming
Some of the positives to the “using” keyword approach are:
- You can manage classes externally from code that defines them, seamlessly adding in functionality by augmenting classes and typedefs.
- Anything you can do through the “using” keyword can be done by using static classes, so there is very little ‘magic’ going on… just some clever reformatting of function calls.
- There’s a ton of static classes out there (in addition to my own) that already contain a lot of good practical functionality. You can easily reuse these classes as “using” classes, or write your own that handle specific needs.
- Method completion works with –display. So once you include the “using” keyword with an appropriate class, you can instantly see the extra available methods by triggering a code-completion request inside your editor (assuming the editor does –display code completion).
I’m assuming “using” will be added in the next version of haXe (2.04). Till then, you can play around with it by downloading and building from the CVS sources.
7/26 Update: “using” is now officially part of haXe (2.04).
self conscious white noise 