By Automating ALL THE THINGS

My summary of a previous talk has most of the links from this presentation, if you don't want the full talk transcript.

Okay, here's the deal.

If you walk out of here with one technique or one automation that you want to try, a change you want to implement in your life, I will consider this talk a success. The trick is going to be, however, how I keep you entertained until you find that trick.

About a month and a half ago, a number of friends asked at dinner time what I was going to do for my new years resolutions. They knew that for the last ten years, I had dutifully made my New Year's resolutions, and, because my birthday is close to the mid-year, my nth year resolutions.

There were a couple of those resolutions were on every one of those lists. Twice a year, I would make a couple of the same resolutions. Twenty times, these resolutions appeared on my lists.

This past new year's, I didn't make resolutions. Not because I didn't have anything I wanted to change, we're all here in a continual process of self-improvement, and I am no exception there. Realistically speaking, however, resolutions are wishes unless they have a action plan associated with them.

Once they have an action plan associated with them, they're not resolutions, they're commitments.

Now, if I make a commitment to do something, not just wish I would do it, dream I would do it, think about doing it, but really commit to do it, I want to set myself up to succeed with that action. I want to commit to it all the way, and in my case, finally follow through on that 10 year old resolution.

So... Who am I and why can I talk to you about setting yourself up to succeed?

Well, my name is Kitt Hodsden, and I'm the 47th laziest person in the world. With this delightful achievement, I have still managed to find success.

Now, wait, successful and lazy? Isn't being lazy a bad thing?

Laziness – The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful, and document what you wrote so you don't have to answer so many questions about it. Hence, the first great virtue of a programmer.

http://en.wikipedia.org/wiki/Larry_Wall

Being lazy happens to be one of the three great virtues of quality programmer, according to Larry Wall. The other two are impatience, which I have plenty of, I can assure you, and hubris, which I'm not so sure about, but hey, let's go with it.

Is this really a good thing?

"Progress isn't made by early risers. It's made by lazy men trying to find easier ways to do something."

Robert A. Heinlein quotes (American science-fiction Writer, 1907-1988)

So, since I'm lazy, but still wanting to succeed, I'm going to be sure to set myself up to succeed. I'm going to streamline and automate away many of the small, and sometimes bigger, tasks, use technology to make my life easier.

The less time I spend fighting my surroundings and my tools to achieve a goal, the more time I have and the more likely to be successful.

So, that resolution? The one I kept adding every year for ten years? What was it?

Stop biting my fingernails.

Kinda gross, huh?

Yeah, I thought so, too.

Given this is a lifetime habit, how did I break it?

The first I did was make it easy not to bite my fingers. I made the action that I wanted, the end goal, what I defined as "success," easy for me to achieve. I set an alarm to trim my nails every other day. If I don't have nails, I can't really bite them.

"Make it easy" is not to say the task itself is necessarily easy, but the act of doing the task should be easy.

On a small scale, when doing work at a computer, daily work is much the same. Make things easy for yourself, become efficient, make the computer work for you.

⌘ space

An easy example of making things easier with modern systems is using an application launcher.

If my hands have to leave the keyboard, I have become inefficient, and have probably missed an opportunity to streamline something. To open applications or files, I hate GUIs and having to use the mouse.

I use flower-space to open up Alfred.

Okay, okay, flower-space, command-space, splat-space, bowen-knot-space, same thing.

Alfred is an application launcher, that gives you access to bookmarks, runs shell scripts, or AppleScripts or Automator scripts. I can open files in different applications without having to move my hands from the keyboard.

This is great!

Small efficiencies that build up into an easier workflow!

This is great! On the practical side of things, I’ve already made things easier for me.

On the mac, there's also Quicksilver and Launchbar.

How many people here use mainly Windows systems? How about Linux?

Okay, if there were windows people in here, I would talk about the windows options, which include AppRocket and Launchy. Launchy happens to have a python plugin where you can expand what it does with python scripts. Once you have scripting access, the world opens up.

Skylight
http://www.candylabs.com/skylight

Launchy, which gives you Python hooks
http://www.launchy.net/
http://pylaunchy.sourceforge.net/docs/

For Linux people, I'm told that Gnome Launch box or Gnome Do have similar functionality and extensibility:

Gnome Launch Box
https://live.gnome.org/GnomeLaunchBox

Gnome Do
http://do.davebsd.com/

If you move across operating systems regularly, Launchy happens to work across all three of those operating systems. Use it if you would like a consistent interface.

Okay, so why again am I so excited about an application launcher?

Because it is more than just an application launcher. If you let it, it becomes command central.

You can set up shell scripts or AppleScripts, create a workflow with Automator (I did mention this was a Mac centric talk, for which I apologize somewhat), or create a group of applications that open automatically for a specified command you define in Alfred.

So, first step, small that it may be, is to make things easy for me.

Let's talk about making things even easier, because, hey, we're working towards that title of Most Lazy Person in The World, which could take a little effort, is to learn to love the command line.

Loving the command line opens up a whole big world of automation it gives you the ability to chain together disparate programs and applications, and glue them all together. You can use cron and at (coming back to these later), you can write scripts in your favorite language and run them in the terminal, hell, you can trigger the end of the world no less. To be efficient and make things easy, you want to be comfortable in the command line.

Most importantly, you can keep your hands on the keyboard, which, recall helps in the efficiency of all the things.

If you have never used the command link or shell, know that learning the command line is doable. Fifteen command will get you going:


cd
cp
less
ls
pwd

rm
cat
find
grep

!!
!$
which

pushd
popd
dirs -v


These enable you to move around the file system, know where you are, look at files, and find files.

If you have used the command line before, pay attention to those last three commands.

If you use the GUI, see if you can run your programs on the command line. Revision control systems such as svn or git have command line interfaces. You can interface with a large number of websites via their APIs through the command line. You can upload images, read discussions, tweet, post comments, generate assets, create CSS, all sorts of fun stuff from the command line. And you can make things easy by automating much of your work, starting with the shell.

Command line tutorial resources
http://quickleft.com/blog/tag/bash
http://rubytune.com/cheat

There are many resources for learning the command line. Very much worth your time if you're not already.

Let's take as an example, the need to upload a picture or several to a webservice or image gallery. To add an image to an album, I could open up the web page, log in, go to the upload window, select a file, hit submit, wait for the URL, copy it to my clipboard, then maybe tweet it.

If you use Flickr, you can upload it from the command line:
http://www.flickr.com/services/apps/tags/commandline

If you use Trovebox:
https://trovebox.com/documentation


dymock[1037]% openphoto-upload all-the-things.png


Or let's say, I have my own webserver, I have used ImageMagick tools to convert formats and resize an image. I can use the command line and copy it to my server instead of using an uploader:


dymock[1042]% convert all-the-things.jpg all-the-things.png
dymock[1043]% scp all-the-things.png kitthodsden.org:kitt-images
all-the-things.png 100% 26KB 25.6KB/s 00:00
dymock[1044]%

That's much faster than the webpage access, and, once I'm used to it, easier than web interface version.

Even if you're a beginner on the command line, in your terminal, watch to see what kind of work you do on the command line. Once you do something the same more than three times, set up an alias.


history | cut -c8-120 | cut -d" " -f1 | sort | uniq -c | sort


Aliases are pre-defined short cuts, a way to add custom commands to your shell.

Back to my convert the format and upload my picture to my server:

Typing that out is going to be a pain in the butt every time I want to do it. So, I set up an alias.

dymock[1042]% convert all-the-things.jpg all-the-things.png
dymock[1043]% scp all-the-things.png kitthodsden.org:kitt-images
all-the-things.png                  100%   26KB  25.6KB/s   00:00

There are a number of ones I like to use that are the bare minimum I copy to every server. Most people tend to grow their list of aliases over years. Store them in your profile or resource file, and keep a git repository of your various environment files is wise, enabling you to deploy to new systems easily.

zsh, bash format
alias d="dirs -v"

tcsh format 
alias d 'dirs -v'

A website that has a number of aliases, in a communal sharing way, is alias.sh. I've found a number of new aliases on the site just browsing. Good resources.

As an aside when speaking about aliases, there are two things I find frustrating when I'm working with an experiened on the command-line friend navigate the command line slowly.

The first is using arrow keys to move the cursor to the beginning of the line or the end of the line. Control-a, to the beginning of the line, Control-e to the end of the line (if you are on a mac, those work in just about every application including the URLs textbox of browsers), Control-r to reverse search, Alt-b to go back a word, Alt-f to go forward a word - unless! you have vi bindings in your shell, at which point, well, you're customizing your shell, so you're doing okay.

The second most frustrating task I watch people do in the terminal is use cd to change file directory locations back and forth. I can't stand that. If I'm trying to help you, and you do this, I will walk away from you.

cd /home/takethelongroad/path/to/interesting/location
cd ../../another/interesting/location
cd ../../../off/to/different/place
cd ../../../../another/interesting/location

A number of items to fix that:

When you use the terminal, use aliases for your common locations

alias biz 'cd ~/workspace/thepearl/www/sites/all/modules'
alias thn 'cd ~/workspace/thundercat/www/'

Much shorter!

Use cd - to alternate between two directories

% cd - 

Depending on your shell, you can use ^r to search through your command history

The best to use is pushd and popd.

pushd, short for push directory, and popd, short for pop directory, gives you a list (a stack, for developers) of directories that you can view with the dirs command.

You use pushd to swap the current directory with the next directory in the stack, or you can specify how deep into the stack you want to pull out of the stack with a pushd +2

You can use the command dirs to view the directory stack. I, of course, am lazy, os I have them aliased to two letter commands.

alias pu pushd
alias po popd
alias d "dirs -v"

alias puhsd "pushd"

Quickly now, other common aliases I've seen from my friends include

cd ..
..

ls -al 
lsl
ll

mkdir -p
md

psg
ps -ef | grep !$ 

hg
history | grep !$

find . -name $*
ff

And one from a friend:


alias gen_ctags='/usr/local/bin/ctags --langmap=php:.engine.inc.module.theme.php.install.test.profile --php-kinds=cdfi --languages=php --recurse --exclude="\.git" --exclude="\.svn" --exclude="\.hg" --exclude="\.bzr" --exclude="CVS" --totals=yes --tag-relative=yes --regex-PHP="/abstract\s+class\s+([^ ]+)/\1/c/" --regex-PHP="/interface\s+([^ ]+)/\1/c/" --regex-PHP="/(public\s+|static\s+|abstract\s+|protected\s+|private\s+)function\s+\&?\s*([^ (]+)/\2/f/"'


Sometimes aliases aren't enough. And, really, that's when everything opens up.

You can write scripts to do anything. They are just small programs that link everything. From the command line you can load web pages, start programs, grab the output from one program (say, that webpage page you just loaded), find something interesting in it, and pass it along to another program.

The only difference between the javascript you use in the browser and the scripts that run on the server is the language, and if you use node.js on the server, even that can be the same.

There are a lot of resources for scripts on the web. The problem with going into depth with scripts, however, is that the examples tend to be very specific to the problem that is being solved.

I have a script that exports data from an Access 2000 database, modifies it, sends it over to another server, imports it into a MySQL database which replicates it to other servers. Yes, that seems like a very odd scenario, I agree, but there were legacy systems and uptime requirements in place that necessitated that script's existance. I would bet a very large sum of money that none of you would have need of that script.

So, how about a reasonable example?

To upload all of my photos using openphoto / trovebox:

So, if I have OpenPhoto installed, and the php command-line option, I can run a script to upload all of my photos:

https://github.com/openphoto/openphoto-php
https://gist.github.com/1274061

openphoto-php-upload-cli.sh

#!/bin/sh

source secrets.sh
for i in $(ls /path/to/directory/*.JPG) ; do
  ./openphoto -h yourhost -e /photo/upload.json -F "photo=@$i" -X POST;
done

Scripts are just a series of logic steps. Programming is just a series of logic steps. Do this, do that, do this a number of times, if that happens, do this other thing. Scripts don't have to be complicated; they can do just one thing.

So, phew!

That's how we can start to make things easy. Little bit of effort to notice what we do repeatedly in our daily workflow, and make them a bit easier with aliases and scripts.

Got that "make it easy" part done. What's next?

Next step to setting yourself up to succeed is to form habits (habits that you want, btw), not the bad ones that you'll spend a decade trying to break!).

With our digital work, triggers work off the magic of cron and at.

cron is your friend
at is your buddy

cron is a program that runs other programs on a regular schedule, as frequently as every minute. If you are on windows, Look at the Windows Task Schedule and the AT command.

When using cron, you specify what programs you want to run and what time with the crontab command:

% crontab -e

Your crontab file is a list of commands with the time you want to run them.

You have options for minutes on the hour, hours of the day, dates of the month, days of the week and months of the year as options. You can specify what command to run, pipe several commands together,

# m h  dom mon dow   command

# every seventh minute, every day
*/7  *  *  *  *  /usr/bin/example -arg

# run ping and ls at noon and midnight on the 1st day of every 2nd month
# output the commands into the log file /var/log/cronrun.
0 0,12 1 */2 * /sbin/ping -c 192.168.0.1; ls -la >>/var/log/cronrun

# third monday of every month at 4 am
0 4 15-21 * 1 /command

If you have a single job to run, you can use at instead of cron. The format is a little easier, and you can type the commands in at the command line prompt.

% at 1500 feb 14
at> ~/bin/tweet "I am now speaking at #webstock about the "at" command. I hope I timed it well."
at> ^D

As an exercise for the reader, if you want to infuriate your system adminstrator, write a self-submitting at job.

They hate those.

So, we want to form habits. Outside of cron and at, how can we do that?

BJ Fogg @bjfogg has this incredible program for developing habits to change behaviours.

http://tinyhabits.com/

In the program, we learn that habits can be formed by triggering specific actions off an anchored routine. When I do X, I will do habit Y immediately after. Starting off small, triggering the event, and celebrating the moment when you complete the action are the essence of the program.

This triggering of desired, good action is another way we can set ourselves up for success.

That habit I was trying to break, the nails? The action of bringing my hand up to my mouth, now triggers the desire to pull out nail clippers, find a bathroom, and clip my nails. If I have no nails to clip, the nail clippers go back in my pocket.

Another action I do regularly that I like to trigger off is logging in to my system. That act triggers my self-improvement action of minimizing any kind of struggle with my tools by learning them.

There is no substitute for using and learning your tools. You can read a hundred books about Whatever-You-Want-To-Learn, but until you start using it, the muscles won't be the same.

A site that helps you build muscle memory by focused repetitions is shortcutfoo.com.

ShortcutFoo is a drill site that that reinforces the shortcuts and builds muscle memory.

A few minutes every day to build those up, and you’re on your way.

And you want to learn how to do macros with your editor

Here's how to record a macro

After defining it once, I can run it again with Control-x e

control-x e

I can run it N number of times with Control-u

control-u N control-x e

I can save that macro and run it again tomorrow, next week, next year.

No, it's not intuitive. Yes, it is waaaaaaaaaaaay worth it.

If your editor doesn't have macros, choose a different editor. Seriously, who wants to do a 16 stage cut-move-cursor-text-replace when you can create a macro once and run it. Most modern editors have a save-macro feature. If yours doesn't, there are plenty of editors that do.

That editor doesn’t have to be a text editor, btw.

Photoshop has Actions, you can use Smart Objects, too.

The act of starting up a program could trigger viewing the menus of the program, poking around, learning short cuts that way. Who knows all the shortcuts of the programs they use?

Yeah, me neither.

If you walk out of here with nothing else, walk out of here with the intention of becoming proficient in your editor of choice, your tools of choice.

While you’re at it, learn your operating system shortcuts!

[notes to myself here: * take a breath * * move around *]

So, to set ourselves up for success, we're going to make things easy, and we're going to use triggers to form habits. Part of this process is intergrating our changes into our lifestyle, making it easy to keep going with these steps to success.

In the last couple years there has been a resurgence of barefoot running. The style of running boasts reduced injury rates, which makes it appealing to a lot of runners, and nearly all injured runners. target="_blank">CHRISTOPHER McDOUGALL wrote an article in the NY Times about the style a couple years ago, and mentioned a certain Walter George.

Walter George was a champion runner who, nearly a century and a half ago, developed the 100-up training technique.

(start showing the technique)

Yeah, I'm doing it poorly, but you can see, it's essentially running in place.

See, George was a "chemist’s apprentice in England and could train only during his lunch hour." He wasn't able to dedicate hours to training, so he focused his resources and time and worked out a technique that worked with his lifestyle.

This is another part of setting ourselves up to succeed.

Making something easy and using a trigger will help, but integrating the changes into your lifestyle keeps it easy. And that's how things start to build.

The practical side of this is to use hooks.

If you use git or svn or another revision control system, then you may be aware of hooks. Hooks are entry points into a system, they let you add additional processes into the work flow. The great thing about hooks is that they PUSH the changes to you. You don't need to continuous check, "was there a change? was there a change? was there a change?", you can receive a notice that a change was made, and act upon it. They can be the triggers you need.

The simplest example of a using source control hook is to have an email sent when you check in a file. The email can contain the files that were changed, the changes that were made, and the check in message.

If you use a bug or feature tracker, you could include the tracker number that addresses the changes you made, and have your check-in message automatically post to the bug tracker.

If you use a continuous integration server, you can have the commit trigger the running of your unit tests.

This is a list of a newly initialized git repository. The files here are sample files that show you when in the source control process you can adjust the flow to you needs.

Each of these hooks are modified by adding scripts that run on the specified input information such as the changed files, the changes made and the message. You can adjust the output before a commit is actually made, or after a check-in is complete to kick off an other process.

This all ties back to the beginning when I mentioned that scripts are the glue that holds the world together. If you can think through a series of steps logically, "If this, then do that", you can write a script.

While you can create hooks to process code during the commit process, use of these hooks don’t apply to only code.

If you have images in your repository, you can trigger ImageOptim, an image optimizer which reduces the image file sizes.

If you are using Compass as your CSS proprocessor, you can create sprites automatically using Compass.

https://github.com/averyvery/FED-Starter-Kit/blob/master/config/compass.rb
http://viget.com/inspire/stop-making-sprites-compass-sass-and-png-sprite-generation

You can kick off your unit tests, enforcing code styles, build your jar file, update your documentation, build your test app, send your files your staging server, to a remote server, tweet your build status, all sorts of automatic actions triggered from the source control check in. That stage really is a great place for integrating work into your digital workflow.

And speaking of unit tests...

So, I have my application that I'm building. If it's a web application, I'll start by testing, I'll start by testing the pages I've created, and as best I can make sure the interactive part works as I expect it to work. When things are small, I'm just clicking through the various parts of the page, triggering various javascript events, watching what happens. Eventually, I'm going to grow tired of filling in that same form a dozen times with different data each time to manually test validation works correctly.

So, after the nth time I do that, blech, yeah, I'll write a bookmarklet that auto-fills in the forms and triggers those javascript for me. One click, form filled in, and submitted:

http://www.phpied.com/form-auto-fill-bookmarklet/

Bookmarklets work just as well for the local case as they do for saving bookmarks to services, sending images to file sharing sites or the like.

So, yeah, this works well for the individual case, but if we're trying to integrate the clicking around a website to test it, and we happen to be on a larger team, and have the resources available, setting up a Selenium or Windmill installation could save time in the long run.

Selenium and Windmill are web testing tools. The part that I happen to like, even for development, is the recording tool. Usually used for automating QA testings, you can also use them during development if you have an app workflow to test. You can also play a test back in different

You start recording, click through the application, then stop recording, and save the playback. You don't *need* to put them as your final QA tests, but watching the tests, or watching the tests that someone else has created, is sometimes insightful.

You're already clicking around on a site during development, recording the information and including it in the regression tests. So, maybe consider this your lifestyle change.

We'll call it 100-clicks.

As another aside, you can use the bookmarklet technique for a lot of other uses. I have a number of them that I use to test my @media queries values by opening windows of different sizes.

The code is fairly simple...

Drag to the toolbar and click on that new link, and it pops open the window in the new size.

I try to have the windows open in slightly off sizes, where the design looks like it needs to scale up to the next size, and the device resolutions, but your mileage may vary on that detail.

So, we have make it easy, form habits, use triggers and integrate what you're doing into your lifestyle, digitally and otherwise.

Walter George was an impressive runner. He integrated his training into his lifestyle. He developed his 100-up training style. There was another trick in his bag, though, one that requires noting.

We also have to do our actions consistently.

You can’t cram for fitness. You can’t cram for skill. Consistency is the road to both.

Consistency in a development workflow includes things like a Continuous Integration system, such as Hudson or Jenkins, where all of your tests can run, automatically.

Consistency in a development workflow includes things like a Continuous Integration system, such as Hudson or Jenkins, where all of your tests can run, automatically.

And you know at a glance if your tests are passing.

If you are color blind, or a designer, this page kinda sucks.

You can change websites to look like what you want them to, on a local level.

Greasemonkey is a plugin for Firefox that lets you use the Firebox browser as an application framework of sorts.

You install the Greasemonkey plugin, then install javascript scripts that act up on pages that, well, you don't own and wouldn't normally be able to modify. Except in this case, now you can!

As always, when you are installing software on your system, check it over before actually installing it. If you can't write your own scripts, you can find community contributed greasemonkey scripts on http://userscripts.org/.

In this particular case, with the Jenkins output, if you prefer them orange and blue, for example, you can look at the HTML, find the CSS that you want changed, and write a quick javascript file that, on load, adjusts the CSS classes.

Now, every time I go to view this page, the CSS automatically changes, and I see the colors I want to see.

Or maybe the ones you don't want to see.

This is, by the way, how I automated playing a Facebook game five years ago.

It required loading and reloading a page over and over again, and every 100 to 500 page reloads, a coin would appear. I set up a script to reload the pages and pause when a coin appeared on the page.

That game was so much more fun to play that way.

Consistency can also mean dedicating time each day to learning your tools.

Or integrating learning ways to speed up your existing processes by being aware of them.

There are times when my IDE or editor just doesn’t have the functionality I need.

Emmet, formerly zen coding, is and editor plugin that generates HTML by specifying just tags, ids, classes and quantities.

div#banner>div.logo+ul#navigation>li*4>a

This line will output:

<div id="banner">
  <div class="logo"></div>
  <ul id="navigation">
    <li><a href=""></a></li>
    <li><a href=""></a></li>
    <li><a href=""></a></li>
    <li><a href=""></a></li>
  </ul>
</div>

As an easy example. Emmet does a lot more, vendor prefixing, gradients, actions, and super charges an editor with work with CSS

Or new tools altogether.

http://coursera.org/

So, to set ourselves up to succeed, we're going to make things easy for ourself. We're going to use a trigger. With those triggers, we'll form habits. This will help integrate it into our lifestyle, our workflow, and we'll do it consistently. Doing these steps will set us up to succeed.

Make it easy.
Use a trigger.
Form habits.
Integrate it into your lifestyle.
Do it consistently.

But.

There's one thing you have to do for all of this to work. None of this will do you any good if you haven't already defined what success means to you.

That, you'll have to figure out for yourself.

I would like to thank you for spending your time with me. I really appreciate your attention. If I didn't manage to come up with a single idea on how to set youself up to succeed by automating your work, contact me. I bet we can figure out something for you to try.