Clear - Node Module

Posted by Dave Eddy on 15 Aug 2012

Clear the terminal screen if possible

I wrote up a super small module to mimic clear on the command line for Node.js. I was surprised when I didn't find a module that already did this in npm.

View the Project page on GitHub

Usage

var clear = require('clear');
clear();

Example

Node Clear

clear([bool])

You can optionally give clear an argument of false to prevent it from clearing the screen. This will not remove anything from the screen, but instead move your cursor to position 0,0. Much like printing a \r instead of a \n to reset the current line of output.

Installation

npm install clear

ANSI Codes

http://www.inwap.com/pdp10/ansicode.txt


Musicnamer

Posted by Dave Eddy on 04 Aug 2012

Organize your music collection

I wrote this module as a way to help organize my collection of flac and mp3 files. It can automatically inspect your music files metadata tags, and rename the files based on them.

Check out musicnamer on GitHub

Description

Rename music files to clean filenames based on their music tags. By default, musicnamer can take your music files and rename them to a format like:

%artist%/%album%/%trackno% - %title%.%ext%

This package is not meant to be used as a Node module, but rather as a command line tool

Usage

Invocation of musicnamer is simple: just run the program with files as arguments and it will do its thing on them.

~$ musicnamer -h
Usage: musicnamer.js file1.mp3 file2.mp3 file3.mp3 ...

Given a list of files from the command line, rename them to
a clean filename (default '%artist%/%album%/%trackno% - %title%.%ext%')


Options (must be given as the first argument; all options are mutually exclusive)
  --init    | -i: Create a config file at ~/.musicnamer.json
  --dry-run | -n: Don't actually rename files, just print what actions would be taken
  --tags    | -t: Just print the tags from the files processesd, assumes --dry-run
  --version | -v: Print the version number and exit
  --help    | -h: Print this message and exit

Examples

To invoke musicnamer, simply pass a file over the command line as an argument

dave @ [ bahamas10 :: (SunOS) ] ~ $ musicnamer somesong.mp3
Error reading /home/dave/.musicnamer.json -- invoke with --init to create this file

----- processing somesong.mp3 -----

Moving: somesong.mp3
->  To: BEING/Arrival/12 - The Singularity (Cosmists II).mp3

Music namer renamed the file for us. What effectively happened here is this

$ mv somesong.mp3 "BEING/Arrival/12 - The Singularity (Cosmists II).mp3"

NOTE: musicnamer by default renames files relative to your current directory.

We can see the error message above complaining because the config file was not found/ unreadable. We can fix this warning with this:

dave @ [ bahamas10 :: (SunOS) ] ~ $ musicnamer --init
Writing config to /home/dave/.musicnamer.json

More details on the configuration file can be found in the Configuration section below.

Now let's say we wanted to rename a bunch of files, but were worried about modifying them without testing. You can run musicnamer with a dry run option to show what action would have been taken.

dave @ [ bahamas10 :: (SunOS) ] ~ $ musicnamer --dry-run somesong.mp3
----- processing somesong.mp3 -----

Moving: somesong.mp3
->  To: BEING/Arrival/12 - The Singularity (Cosmists II).mp3
No action taken

As you can see, the warning message no longer shows because we have created a config file. Also, musicnamer just printed out what it would have done, but didn't actually call rename(2) on any of the files.

You can also test out files to get a glimpse into how musicnamer sees your files. There is a command line switch to have musicnamer print out the tags of files without renaming them.

dave @ [ bahamas10 :: (SunOS) ] ~ $ musicnamer --tags music/*.mp3
----- processing song1.mp3 -----

{ title: 'Stimulus',
  artist: [ 'The Omega Experiment' ],
  albumartist: [],
  album: 'The Omega Experiment',
  year: '2012',
  track: { no: 2, of: 0 },
  genre: [],
  disk: { no: 0, of: 0 },
  picture:
   [ { format: '浩条⽥灪来cover\u0000',
       data: <Buffer 00 ff db 00 43 00 02 01 01 01 01 01 02 01 01 01 02 02 02 02 02 04 03 02 02 02 02 05 04 04 03 04 06 05 06 06 06 05 06 06 06 07 09 08 06 07 09 07 06 06 08 ...> } ] }

----- processing song2.mp3 -----

{ title: 'Motion',
  artist: [ 'The Omega Experiment' ],
  albumartist: [],
  album: 'The Omega Experiment',
  year: '2012',
  track: { no: 3, of: 0 },
  genre: [],
  disk: { no: 0, of: 0 },
  picture:
   [ { format: '浩条⽥灪来cover\u0000',
       data: <Buffer 00 ff db 00 43 00 02 01 01 01 01 01 02 01 01 01 02 02 02 02 02 04 03 02 02 02 02 05 04 04 03 04 06 05 06 06 06 05 06 06 06 07 09 08 06 07 09 07 06 06 08 ...> } ] }

This output is good to look for debugging information, without making any modifications to the filesystem.

Configuration

musicnamer --init will create a config file in ~/.musicnamer.json. This config file has a key called format, which has the format to use when renaming files.

~$ cat ~/.musicnamer.json
{
  "format": "%artist%/%album%/%trackno% - %title%.%ext%"
}

The --init option will write out the default format value to the config file, this is the format that will be used if the config file is not present.

Possible options for variables are:

%artist%  : artist name
%album%   : album name
%trackno% : track number
%title%   : track title
%ext%     : file extension

Installation

npm install -g musicnamer

Credits

License

MIT License


Chromium Install Script for Mac

Posted by Dave Eddy on 25 Jul 2012

Install and upgrade chromium on OS X.

Chromium

I like to have both Chromium and Chrome installed on my mac, but installing Chromium is not as easy as installing Chrome. Chromium required me to find the tarballs that were generated from the Chromium source control, and to make sure that I was only pulling the tarball if my Chromium version was out of date. Because of this, mac-chromium was born.

View the project page on github

Usage

Run the script from the command line to download and install the latest Chromium

Examples

Check to see if there is a newer version of Chromium available

$ ./mac-chromium.sh -c
Chromium is out-of-date!
Current Version :: 146768
Latest Version  :: 148329

Download and install the newest chromium for mac

~$ mac-chromium.sh
Chromium is out-of-date!
Current Version :: 146768
Latest Version  :: 148329
Downloading the latest chromium...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 38.3M  100 38.3M    0     0  1432k      0  0:00:27  0:00:27 --:--:-- 3013k
Installed Chromium 148329

Check for an update, exit 0 if up-to-date, 1 if there's an update, 2 if error

~$ mac-chromium.sh -cq

Download and install the newest chromium with no output

~$ mac-chromium.sh -q

Options

-h : print this message and exit
-c : check only, don't upgrade
-f : force an update even if chromium is running
-q : suppress output, just use exit codes

License

BSD 3 Clause


(De) struct - Node Module

Posted by Dave Eddy on 18 Jul 2012

Easily unpack C Structs and binary buffers

I wrote destruct to tackle the problem of unpacking binary buffers in Node easily. Based off the unpack function in Perl, and inspired by prustat by Brendan Gregg, this module makes it simple to take a binary buffer object, and a format string to unpack values. There were other modules that claimed to have the same functionality, but most of them were too poorly documented with the source code almost unreadable, while the others were overcomplicated and didn't support a simple format string.

This module does not handle all data types, it has only been tested on SmartOS, and was built for making it easier to extend the proc Node module by @dshaw.

Check out the project page here https://github.com/bahamas10/node-destruct.git

Usage

var destruct = require('destruct');

Example

var destruct = require('destruct'),
    fs = require('fs');

fs.readFile('/proc/self/psinfo', function(err, buf) {
  console.log(destruct.unpack('iiiiiiiiiiIiiiiSSa8a8a8Z16Z80iiIIaa3iiiiii', buf));
});

yields

[ 33554432,
  3,
  714,
  18456,
  714,
  18456,
  2011,
  2011,
  2300,
  2300,
  0,
  14536,
  9736,
  0,
  1572868,
  11,
  3,
  <Buffer 64 41 f9 4f 3b fe 6c 1a>,
  <Buffer 00 00 00 00 35 d5 31 0b>,
  <Buffer 00 00 00 00 00 00 00 00>,
  'node',
  'node ./psinfo.js',
  0,
  2,
  134511788,
  134511800,
  <Buffer 01>,
  <Buffer 00 00 00>,
  674523,
  3,
  0,
  0,
  29,
  19167112 ]

Install

npm install destruct

License

MIT Licensed


Autocast - Node Module

Posted by Dave Eddy on 17 Jul 2012

Easily and automatically cast common datatypes in JavaScript

I was working on parsing output from the command line while creating the Node SMF module. This required me to inspect the elements individually and write case statements to expect certain data types be returned.

What I mean by this is...

When you parse the command line, everything gets returned as a string. So, you must do some post-processing to cast items that look like numbers to numbers, strings that look like keywords to keywords, ie 'false' to false, etc.

So, autocast was born. This module will do all of that inspection for you and return what it thinks the object should be.

Check out the project page here https://github.com/bahamas10/node-autocast

Example

var autocast = require('autocast'),
    x;

x = autocast('5')
// x => 5
x = autocast('5.8')
// x => 5.8
x = autocast('5.8.8')
// x => '5.8.8'
x = autocast('null')
// x => null
x = autocast('true')
// x => true
x = autocast('false')
// x => false
x = autocast('normal string')
// x => 'normal string'

Install

Install locally to use as a module

npm install autocast

License

MIT Licensed


Realtime DTrace Visualization

Posted by Dave Eddy on 12 Jul 2012

Check out my YouTube video of realtime latency graphs using DTrace and gnuplot.

Scripts here https://github.com/bahamas10/realtime-dtrace-visualization

The graph on the left shows read system call latency, and the graph on the right shows write system call latency (syscall::read*:entry and syscall::write*:entry respectively). The bottom portion is a modified oneliner to show files opened by a process (excluding gnuplot) taken from Brendan Gregg's DTrace oneliners.

At 0:45, I induced latency by reading /usr/share/dict/words and writing the contents out to the screen in a 1 second loop. This becomes visible in large spikes on the graph showing write latency, but the read latency graph is almost unaffected.

The X-axis on both graphs show time (ticking every second) in the form of MM:SS. The Y-axis represents the time in milliseconds, and each point is a 1 second average aggregation of the latency.

Because of the nature of gnuplot in this experiment, the Y-axis adapts to the visible metrics, so when the latency is induced, the Y-axis grows from ~.005ms to ~2ms.


Node DTrace Examples on Github

Posted by Dave Eddy on 16 Jun 2012

Working on performance analysis with an interpreted language poses some interesting problems. Fortunately, Node.js has built in support for DTrace thanks to the guys at Joyent. These built-in facilities make it easy to trace Node.js, and profile its time on the CPU to get an in-depth view of the execution of the program. Tools also exist that use DTrace's ability to profile Node.js to create visual representations of the execution stack, such as Brendan Gregg's Flame Graph Tool.

However, when tracking latency from an application-aware point-of-view, it is necessary to inject probes into the code itself, setting checkpoints, or entry and return points. Using the DTrace Provider Node.js Module written by Chris Andrews, I've created a repository of examples on Github showing the DTrace provider in action.

Check out the repository here https://github.com/bahamas10/node-dtrace-examples/.

Function Tracing

Description

An example from the repository shows how to trace function call depth and latency in Node.js with DTrace

Inspired by dapptrace by Brendan Gregg

Terminal 1:

root@Operationss-MacBook-Pro ~ #  ./simple-nested-functions.js
Looping with 5 second intervals... Beging DTracing now! (provider "nodefunc", pid 39677)

Terminal 2:

root@Operationss-MacBook-Pro ~ #  ./func-trace.d
Tracing...
[2012 Apr 26 16:06:04]  --> func-a-entry
[2012 Apr 26 16:06:04]    --> func-b-entry
[2012 Apr 26 16:06:04]      --> func-c-entry
[2012 Apr 26 16:06:04]      <-- func-c-return (24268 ns / 24 us / 0 ms)
[2012 Apr 26 16:06:04]    <-- func-b-return (95626 ns / 95 us / 0 ms)
[2012 Apr 26 16:06:04]  <-- func-a-return (182276 ns / 182 us / 0 ms)
[2012 Apr 26 16:06:09]  --> func-a-entry
[2012 Apr 26 16:06:09]    --> func-b-entry
[2012 Apr 26 16:06:09]      --> func-c-entry
[2012 Apr 26 16:06:09]      <-- func-c-return (20761 ns / 20 us / 0 ms)
[2012 Apr 26 16:06:09]    <-- func-b-return (81974 ns / 81 us / 0 ms)
[2012 Apr 26 16:06:09]  <-- func-a-return (163436 ns / 163 us / 0 ms)

Node DTrace Example Repository

Usage

This repository is broken up into separate folders. Enter any of the folders to find example Node and DTrace scripts using the dtrace-provider.

Installation

Run npm install before using any of the examples.

Contribute

The more examples we can get in here, the better. If you have any good example Node.js or DTrace scripts, fork the repo, commit them, and submit a pull request.


Manpage Resume

Posted by Dave Eddy on 03 Feb 2012

My resume was getting a little dated, and needed to be updated badly. I've gotten sick of the current style of the page (which was done all in HTML/CSS) and wanted to change it up. The result of that is a manpage formatted resume.

The page will automatically redirect to the pdf version of the resume, but if you would like to get the full experience of my resume in your man viewer, run these commands.

wget http://www.daveeddy.com/resume/daveeddy.7
man ./daveeddy.7

As a bonus I have pushed out my resume to all of my servers using puppet, so on any one of my machines I can issue a man daveeddy

man daveeddy


Acer Aspire One + Xubuntu

Posted by Dave Eddy on 03 Feb 2012

Acer Aspire One

A couple of weeks ago I bought an Acer Aspire One! It never even made it to the "Welcome to Windows" screen; it was boot up with an Ubuntu USB key. I tried desperately to get used to gnome 3, and unity, and I just couldn't do it. Besides painfully slow, I'm just not a fan of fullscreen apps, dynamic desktop space allocation, lack of customizability, etc.

After a couple days of that I decided to try out Xubuntu. I wanted to get a Linux experience I was used to, but have a window manager that I actually like using. Xfce has turned out to be amazing! it's extremely light weight, and it has the full customizability that I loved from Gnome-2.

I upgraded the ram from 1G to 2G, and upgraded the internal 250GB hard drive to a 60GB SSD. After doing some tuning in the system to disable atime, use a different queue scheduler for the disk, and offload the chrome cache directory into a tmpfs, this thing flies. It's still a little difficult getting used to the small size of it.. but i love it.


Scripts Repo

Posted by Dave Eddy on 30 Jan 2012

Check out my scripts repository on github. I have written a bunch of random scripts that needed a home, so I put them up on github. These scripts are put on all of my servers in my ~/bin folder, controlled through puppet. In the repo you'll find a flac-to-v0 script that acts just like cp, in that you give it 1 or more source files, and a single destination, and it will convert the given input files (as flac) to mp3. I also have a php rss parser, a shoutcast/di.fm parser/player, and other random stuff, check it out!

https://github.com/bahamas10/scripts