teleaoidos
from a great distance, singing about code
August 8, 2014
Code Journal 8/7/14 — internetcolor time is on github, MVP status
Rewrote tweet-parsing. Working now. Getting mode of tweet. As most popular colors are black or white, also storing counts of other colors in my four-second interval in my imaginatively named ‘colorModeObj’ object. Some text analysis, good stuff. Working now on client end, making sure everything is streaming right and that I’m parsing correctly there. Hope to have some graphics up and running by the end of the night!
——
Success! My colorstamp object is streaming correctly and I’m pulling timestamp data from it in the console!
Further success! I can pull all colors tracked over a four second period, and the count for each color, from the client. My data is parsing and streaming correctly to the client, and being interpreted correctly client-side! Now, to build the graphics on the client….
—
…and more success: https://github.com/adamcahan/internetcolortime
Am working on getting it set up in Heroku or something like that.
It is short on all the planned features, but it’s a working MVP. Hooray 🙂
Also, pretty nice…..
http://stackoverflow.com/questions/1053843/get-the-element-with-the-highest-occurrence-in-an-array
August 8, 2014
Code Journal 8/2/14 — Mixins, Digging into Express a bit
WHAT IS A MIXIN?
‘…a class which contains a combination of methods from other classes.’
According to Wikipedia this should not be done by inheritence
Angus Croll is the man.
In Express 4, app.’s has a ‘mixing’ object which uses the utils-merge package. Based on a quick read of utils-merge @ npmjs.org utils-merge is not a ‘true’ mixin, or at least a different implementation than described in Angus’ post above. Because utils-merge literally merges objects – variables, methods, and all.
I am attempting to gain a better understanding of what happens on require(‘express’), and require(‘http’), and require(‘socket.i’). Since I’ve encountered this issue (getting socket.io to work with express 4), I might as well take advantage of it and learn something.
My current unanswered question? What’s up with app.handle in:
function createApplication() {
var app = function(req, res, next) {
app.handle(req, res, next);
};
mixin(app, proto);
mixin(app, EventEmitter.prototype);
app.request = { __proto__: req, app: app };
app.response = { __proto__: res, app: app };
app.init();
return app;
}
My current best answer? Well, I still get confused sometimes. The handle() function does not actually get called at any point in this code. Because, the createApplication function is only being defined, not called. So most likely there is a handle() function in one of the mixins. That means all will be fine later when createApplications is actually called.
I also want to understand where Express starts up a web server, because that seems to be part of my socket.io issue – I need to get socket.io linked up/listening to a port on a server, but there is a conflict if I’m creating a second server and there are two different ports or something.
This stack overflow post has a good overview of how the server is started.
Based on that, I took a look at mysite/bin/www
This is what I saw:
#!/usr/bin/env node
var debug = require(‘debug’)(‘mysite’);
var app = require(‘../app’);
app.set(‘port’, process.env.PORT || 3000);
var server = app.listen(app.get(‘port’), function() {
debug(‘Express server listening on port ‘ + server.address().port);
});
First we tell Bash were to go to find the node executable to run node from the command line – or something like that.
Then our debug object is set up.
I am not sure where ‘app’ comes from but there’s a good chance it is from Express’s app.’s.
Here’s the doc on it.
And, actually, here is a very nice related post which my solve my express/socket.io issue.
In any case, the app object in the www file clearly starts up a server, because app.listen() sets that server listening on a port.
—————
Alright, success! My web socket connection is now working. I am getting an error reported in the console, but it is either an issue specific to Firefox or something where things are more-or-less OK, but I need to handle when the socket connection closes. Not going to deal with it now as it’s not crucial. Relevant links for future use:
https://github.com/Automattic/socket.io/issues/1123
http://stackoverflow.com/questions/14140414/websocket-interrupted-while-page-is-loading-on-firefox-for-socket-io
August 8, 2014
Code Journal 7/30/14 — Planning my escape…
FOR THE FUTURE –
GOING FROM WP TO A GITHUB BLOG….
http://derekwyatt.org/2014/01/01/moving-from-wordpress-to-github-pages.html
Getting socket.io and Express 4 working, the story so far….
http://stackoverflow.com/questions/23281594/configuring-express-4-0-routes-with-socket-io
http://stackoverflow.com/questions/24222483/socket-io-1-0-express-4-2-no-socket-connection/24222869#24222869
http://socket.io/docs/
http://nodejs.org/api/net.html#net_server_listen_port_host_backlog_callback
RANDOM
July 30, 2014
Code journal 7/29/14 — Socket.io, Javascript’s call() and apply() function methods
>Primarily I need to determine if my server-side emits must occur within the socket.on(‘connection’,function(){/*..*/} call. Also, how to pass my twitter stream to my socket, and if I can have my twitter stream within app.js w/out blocking (I think so, because the function/method calls are all asynch – so that should be fine…
Socket.io Resources
–http://psitsmike.com/2011/09/node-js-and-socket-io-chat-tutorial/
–https://github.com/LearnBoost/Socket.IO-node
-https://github.com/Automattic/socket.i
–http://howtonode.org/websockets-socketio
–http://stackoverflow.com/questions/4094350/good-beginners-tutorial-to-socket-io
–http://danielnill.com/nodejs-tutorial-with-socketio/
–http://sideeffect.kr:8005/
–http://vijayannadi.wordpress.com/tutorials/sample-chat-app-using-nodejs-socketio/
–http://javaguirre.net/2014/02/11/twitter-streaming-api-with-node-socket-io-and-reactjs/
Serving Static content in Express
>Here is a good overview of how Express handles static content, from directory structure to the standard middleware used for caching/compression. Concise and useful.
>Express.static appears to set what Express uses as the root directory when searching for static content. Node’s path module is invoked as the parameter for express.static(). Path is used ‘for handling and transforming file paths’, as the Node.js documentation says. And path.join is used to join strings together to form a file path – path.join(‘/foo’,’bar’) returns ‘/foo/bar’ – note the slashes added by the join() function.
The relevant app.js’s command – app.use(express.static(path.join(__dirname,’public’)) takes advantage of Node’s global variable __dirname, which returns the path to the root directory of the currently executing script (http://docs.nodejitsu.com/articles/getting-started/globals-in-node-js). This means we take the root directory app.js is in -i.e. our project directory, and append to it ‘/public’, and set this in the Express middleware as the first place to use as ‘root’ when looking for static content. So for ‘/Users/dev/www/mysite’(yes I’m on a Mac:)…
app.use(express.static(path.join(__dirname,’public’)) — this sets ‘/Users/dev/www/mysite/public’ as the root when looking for static content (not sure if I’ve described the ‘relative root’ correctly but hopefully you get the idea!). Good stuff. I am appreciating Node more and more.
FUN FACTS
-A reference of escape sequence for customizing your Bash shell prompt
-Javascript’s apply() method. I encountered it in a basic example of how to use web sockets. I am actually going to start by explaining call(), because it is simpler and understanding call() helped me understand apply(). I’m also going to crib pretty heavily from the excellent explanation here.
EXPLANATION TIME…
>So….first, functions in javascript are just object. And all functions come with some built-in messages, such as toString(), call(), and apply(). Furthermore, there is a ‘global’ object. So the following code (unashamedly copied), does…
var x = 10;
function f(){
alert(this.x);
}
f();
>a) This confused me @ first because f() _is_not_called_through_the_instance_of_an_object, and, if it was, I believe we’d get undefined. When we call f(), ‘this’ refers to the ‘global object’ — the big scope, which is where x is defined. So all is good. call() allows us to assign the this pointer for the function in question. So, as in the tutorial, if we added to our code…
var o = {x: 15};
f.call(o);
>…Now, we get the value 15 because call() has assigned the this pointer to ‘o’. Call() allows us to supply arguments to the function in question. Apply() is just like call(), except apply()’s arguments must be an array.
>Here is the code of the example from O’Reilly’s ‘Javascript Web Applications’:
var rpc = {
test: function(arg1,arg2){/* …. */}
};
socket.onmessage = function(data){
//parse JSON
var msg = JSON.parse(data);
//invoke rpc func
rpc[msg.method].apply(rpc,msg.args);
};
…and the JSON sent was: {“method”: “test”, “args”:[1,2]}
… What this gives us, security-wise is
a)We’ve set ‘this’ for our rpc method call to the rpc object, thus limiting the scope of any code that runs and so ensuring any malicious code can’t access anything (hopefully, I’m no security expert…) outside the rpc object.
b) If the data passed us invokes a method which we haven’t pre-defined, nothing happens. Similarly, we do not run a method on incorrectly formatted data. This may not be as crucial, security-wise as item a), but it also gives us some flexibility.
July 30, 2014
Code journal 7/26/14 — VIM Protip
>Got my new swatch space working – yay! Back to the db now….I got sick of manually making all my comments in VIM. I’m not going to mess with installing a comment package now, but I did find a list of good Vim packages to check out:
>What I also found out, vim-wise, which is nice, is “ctrl-d (highlight what you want in visual mode) shift-i (type what you want to insert) escape”. A quick way to insert something @ the beginning of several lines, such as say, ‘//‘ for comments! Not bad. #realprogrammersusevi
>EOD: Finally got my SwatchSpace all set up…had to re-orient coordinate space around center point, not try to start from top left to assign ‘center points’ of grid squares….didn’t get much else done. I did do a refresher on Express which left me confident about tackling the next part: bringing my frontend page into express, and then setting up the middleware (routing, app.use()!) I need to stream (ah yes, with the event emitters – my knowledge is growing :)) the twitter data to the frontend. In some ways the cassandra db may be unnecessary if I end up going completely real time, but I’ve already planned it in and a database storing stuff seems like a good thing to have – if nothing else it will allow a user who just viewed the site at 2pm to see what’s been going on previously.
Resources
–http://mothereffinghsl.com/
–http://stephensugden.com/middleware_guide/
–http://evanhahn.com/understanding-express/
–http://embeddedjs.com/ (used by nom)
–http://blog.liangzan.net/blog/2013/08/12/three-tips-for-managing-complexity-in-node-dot-js-projects/ (organizing node.js for larger projects)
–http://blog.liangzan.net/blog/2013/01/07/how-to-learn-enough-web-design-to-not-look-scammy/ (good web design/design advice)
July 30, 2014
Code journal 7/21/14 — Node.js Interfaces and Modules
References
-Nice layout of the tweet object.
-Beautiful article on Interfaces for Node Modules.…and a much simpler tutorial….and one more tutorial on export
>Some good progress. DB writes are successful in a module. Tested. Haven’t integrated to app yet. Consolidated my code in a directory. Worked on new version on frontend, rewriting SwatchSpace obj, stuck on javascript class/object method constructions stuff and debugging. Really want to get this frontend ready so I can feed a stream to it….
July 30, 2014
Code journal 7/20/14 — Apache Cassandra and Node.js
>Following instructions here on getting Cassandra set up with Node. Some configuring to do. Also using this as a reference to where Cassandra files are on my Mac. Very useful.
>NOTE: Cassandra’s cqlsh shell seems to not play nicely with tmux. Holding off on investigating this for now. Actually, running sudo easy_install took care of that – great!
>Based on some internet research I am going with cassandra-cql as my node package for working with Cassandra. One, because I found a tutorial with it, two, because it doesn’t mess with Thrift (some Apache API thing for making it easier to use a variety of languages working w/Cassandra, I didn’t look at it that much once I saw it’s been superseded by cql), and three, because while there hasn’t been a lot of activity on the cassandra-cql github site recently it looks well-documented and the author works for Datastax.
>An excellent breakdown of Cassandra’s architecture and storage process
July 30, 2014
Code journal 7/17/14 — Installing Apache Cassandra on OS X
>On someone’s advice looked @ Cassandra. Using instead of MongoDB.
Apache Cassandra Resources
–http://www.ibm.com/developerworks/library/os-apache-cassandra/#list2
–http://wiki.apache.org/cassandra/ArticlesAndPresentations
–http://www.datastax.com/docs
–https://www.npmjs.org/package/node-cassandra-cql
–http://msvaljek.blogspot.com/2014/01/hello-cassandra-in-nodejs.html
–http://planetcassandra.org/try-cassandra/ — Good Cassandra Intro tutorials
Installing on OSX
>This guide pretty much nails it. Straightforward Homebrew install plus a little configuration (and Python).
>Ran into some trouble. Had to install JDK version 7, and follow the instructions here.
>My null pointer error is OK, I think! I am good!
http://stackoverflow.com/questions/24120676/cassandra-null-pointer-and-no-information-in-logs
>The Ops Center looks like a nice monitoring tool. I think I’ll have to install it independently – not sure I downloaded the dataStax OpsCenter Cassandra package. Or maybe I did and I just have to find where the executable is living.
–http://www.datastax.com/2012/01/working-with-apache-cassandra-on-mac-os-x
teleaoidos 