I hate it when that happens

My bank just died.

Crud.

Huh?

What kind of insanity is this? McCain's not willing to meet with Spain's prime minister? I mean...what?

Brisbane

My new place:




As the saying goes, it's not much, but I call it home.

Good news

I found this to be oddly heartening: both Obama and McCain seem to have their heads screwed on pretty straight with regard to a wide variety of science-related issues.

Politics ad nauseam

There's one thing to be said for this year's election circus: it's a much more interesting spectacle than any I've seen before. What's also interesting about it is that the candidates are not reaching for the middle: Obama and Biden are both pretty far left, both economically and socially, and McCain's pretty much the personification of national greatness conservatism. The media's made a lot of hay out of McCain's policy shifts (and correctly so, in my view), but his real core's always been about America's awesome, our military is ridiculously powerful and we should use it like a battering ram whenever possible, and also have you heard that I was a prisoner of war because it's not like I mention it six times a day prior to shaving. I'm still trying to figure out exactly what Palin's all about. She certainly seems to lie a lot. You'd think if you were going to introduce yourself to the nation by telling lies, they'd be about things that were not easily verifiable.

GOV. SARAH PALIN: [repeat forcefully 25x] I hate earmarks, America! I told the federal government Thanks but no thanks on the Bridge to Nowhere!

CHARLIE GIBSON: That's actually not true. In fact here's a picture of you literally wearing a T-shirt saying you support it.

GOV. SARAH PALIN: Uh, well, I-

SEN. JOHN MCCAIN: [raging] BARACK OBAMA WANTS TO SHOW YOUR KINDERGARTENER PICTURES OF HIS BLACK COCK, I SWEAR TO GOD! LOOK HOW CREEPY AND BLACK HE LOOKS IN THIS PICTURE AS HE HOVERS OVER VARIOUS YOUNG WHITE CHILDREN! DID I MENTION I WAS A PRISONER OF WAR BY THE WAY BECAUSE I THOUGHT YOU MIGHT NOT HAVE HEARD SINCE YOU'VE BEEN LIVING IN A CAVE, VERY SIMILAR TO THE PRISON WHERE I WAS ONCE BRUTALLY TORTURED BY THE VIET CONG!

...which is more or less what the past week has seemed like to me. Entertaining, in sort of an idiotic, surreal way, and kind of depressing for the same reason. This is our politics, huh. How about debating policy, guys? Anyone? Mention a policy? Joe Biden had some policy tidbits, too bad he fell down an open manhole cover and died, which is what must have happened because why haven't we heard from him in like a week?

That said, what's interesting about the race is that it makes you review your political leanings, since it isn't permeated with the odd feeling that all the candidates are actually the same guy (the guy being the bastard love child of Ronald Reagan and John F. Kennedy, actually a robot controlled by Karl Rove, who is actually a robot owned by Halliburton). When people ask, I tell them I'm 'basically a libertarian.' Kinda. I like the idea of small government, anyway. Or, more precisely:

Economically, I'm in favor of free markets. They're efficient, they're responsive, they're Adam Smith's cold dead invisible hands crushing our lives into meaningless dust. Just kidding. But in general, I think if a job can be reasonably done by private industry, it should be done that way and the government should leave them alone to do it. There are areas that are not handled adequately by private industry, in which case I'm fine with the government doing it. Environmental protection is one really obvious example of this, and one where I differ sharply from orthodox libertarianism. Yes, you can imagine hypothetical scenarios where a private company would want to protect the environment. But there are enough cases where profit and conservation part ways that the free market, left to its own devices, will give the environment a nice firm rogering. The solution isn't to nationalize these industries, as some of the more fringe leftists would suggest, but to regulate them. This is already being done. Tweaking the level at which this is done is fine and probably a good idea, proposing to either massively deregulate everything or nationalize everything is not fine.

Socially, I'm about as liberal as you can get. While I've got a relatively boring personal life, I fully support your right to use all manner of wild and crazy drugs, have sex with and marry whoever suits your fancy, believe in whatever the hell you want to believe in, and so on. I've got mixed feelings about abortion, but I certainly don't think it's the federal government's business to say you can't have one.

Pet hot button issues of mine: science (more funding for all the natural sciences, please, both basic and applied), space (more space development and exploration, not just lip service, and support commercial space initiatives, please), free speech (book banning, censoring, and excessive political correctness are all pretty horrible things, and yes 'speech' on the internet and in video games still qualifies as speech), guns (don't own one myself, but it's really not ok to say that people shouldn't be able to defend themselves, and yes the second amendment really does guarantee that right, so stop pissing on it).

Where does this odd melange of ideas leave me with regard to this year's elections? If you just look at policy details, it's actually kind of a toss-up. The elephant in the room (pun intended) is that John McCain is older than the big bang and has a temper hotter than dirt (OOPS, DID I FLUB THAT LINE? MAYBE IT'S BECAUSE I'M 72 YEARS OLD AND SENILE AND HAVE NO BUSINESS RUNNING FOR PRESIDENT BECAUSE I HAVE ONE FOOT IN THE GRAVE AND THE OTHER IN MY MOUTH BECAUSE I'M SENILE AS HELL), and regardless of his particular policy stances, really isn't temperamentally suited to be President. Frankly, I'm not sure Sarah Palin is, either, although in her case I'm willing to chalk it up to ignorance. Not that that's much better. Perhaps we should go to war with Russia? Not even the most diehard supporters of the Bush doctrine think that. Not that Mrs. Palin would know.

So, damning him with faint praise, I'm supporting Obama. I do like his idea of changing the tone of politics in Washington. Too bad the McCain camp ruined that by burying him and his change of tone underneath a giant stinking mound of political feces. Thanks for nothing, assholes.

Training, redux

After an epic amount of debugging, I have my logistic-function neural network working! Because I'm going to be using it to process a lot of large training sets, speed was a priority, so rather than using Python, I wrote it in C++ (a language with which I was marginally familiar...i.e., a lot of the 'debugging' time ended up being devoted to learning C++). Coding in C++ makes me realize just what a joy Python really is. If you're used to Python's wonderful lists, doing anything that involves passing arrays in C++ will feel like absolute hell. (Strict data-typing alone is at least a minor stint in purgatory...)

This algorithm is nice because it's scalable (can easily set the number of layers, and number of neurons per layer, to be any size). The listing is below; here the network is just learning an XOR logic. Since I built this from the ground up, it shouldn't be too hard to hook the program up to the genetic-circuit transfer function predictor I wrote this summer, and use it to optimize the predictor's input parameters for a particular target. I need to translate the predictor's Matlab script into C++, as well, and bundle that with my implementation of the Runge-Kutta ODE solver, which should be fairly straightforward.

/*********************************************************************
 * neuralnet.cpp                                                     *
 * Logistic function neural network (compile with Mersenne-Twister   *
 * random number generator: mtrand.cpp)                              *
 * (c) Pericles v. 2.0, 9/9/2008                                     *
 *********************************************************************/

#include <iostream>
#include <cmath>
#include <vector>
#include "mtrand.h"

using namespace std;

const int MAX_CYCLES = 100000;
const int NUMBER_OF_LAYERS = 3;
const int NEURONS_PER_LAYER = 4;
const double GOAL[4] = {0.0, 1.0, 1.0, 0.0};
double INPUT_LIST[4][2] = {{0.0, 0.0},
                           {1.0, 0.0},
                           {0.0, 1.0},
                           {1.0, 1.0}};

// Seed random number generator with system time
MTRand_open mt((unsigned)time(0));

class Neuron
{
private:
 vector<vector<double> > input;
 double * output, * grad, * weight;
 double lRate;
 int numInputs, numSets, layer;
public:
 Neuron() { }
 // Empty constructor: use initializer for neuron construction
 void initialize(int L, int N, double R)
 {
  layer = L;
  numInputs = N + 1;
  numSets = 4;
  lRate = R;
  // Initialize a numSets x numInputs vector of zeros for input
  vector<double> numPerSet(numInputs);
  output = new double[numSets];
  grad = new double[numSets];
  for(int i = 0; i < numSets; i++)
  {
   output[i] = 0.0;
   grad[i] = 0.0;
   input.push_back(numPerSet);
   // Insert static threshold (bias) of -1.0
   input[i][numInputs - 1] = -1.0;
  }
  // Initialize weight to random values on (-1, 1)
  weight = new double[numInputs];
  cout << "++ Layer " << layer << " neuron: [ ";
  for(int i = 0; i < numInputs; i++)
  {
   weight[i] = (2 * (mt() - 0.5));
   cout << weight[i] << " ";
  }
  cout << "] ++\n";
 }
 void forward(int);
 void updateDeltas(int);
 void updateWeights(int);
 double sigmoid(double x) { return 1.0 / (1.0 + exp(-x)); }
 // Functions to access and return member data
 int getNumInputs() { return numInputs; }
 int getNumSets() { return numSets; }
 double getOutput(int i) { return output[i]; }
 double getGrad(int i) { return grad[i]; }
 double getWeight(int i) { return weight[i]; }
};

///////////////// Network declaration and sizing /////////////////

vector<Neuron> layers(NUMBER_OF_LAYERS);
vector<vector<Neuron> > network(NEURONS_PER_LAYER + 1, layers);

//////////////////////////////////////////////////////////////////

void Neuron::forward(int inputSet)
{
 switch(layer)
 {
  case 0:
  {
   double (* pTemp)[2];
   pTemp = INPUT_LIST;
   for(int i = 0; i < numInputs - 1; i++)
    input[inputSet][i] = pTemp[inputSet][i];
   break;
  }
  default:
  {
   // Use the previous layer's output as this layer's input
   for(int i = 0; i < numInputs - 1; i++)
    input[inputSet][i] = network[i][layer - 1].getOutput(inputSet);
   break;
  }
 }
 // Calculate the neuron's output from the weighted inputs
 double product = 0.0;
 for(int i = 0; i < numInputs; i++)
  product += weight[i] * input[inputSet][i];
 output[inputSet] = sigmoid(product);
}

void Neuron::updateDeltas(int inputSet)
{
 double error;
 switch(layer)
 {
  case NUMBER_OF_LAYERS - 1:
  {
   // Calculate how far off the output is from the goal, and
   // adjust the hidden-to-output weights
   error = GOAL[inputSet] - output[inputSet];
   grad[inputSet] = error * output[inputSet] * (1 - output[inputSet]);
   break;
  }
  default:
  {
   // Backpropagate the error and adjust the previous-to-current
   // layer weights based on the gradients of the next layer
   error = 0.0;
   for(int i = 0; i < network[0][layer + 1].getNumInputs(); i++)
    error += network[i][layer + 1].getWeight(i) * network[i][layer + 1].getGrad(inputSet);
   grad[inputSet] = error * output[inputSet] * (1 - output[inputSet]);
   break;
  }
 }
}

void Neuron::updateWeights(int inputSet)
{
 for(int i = 0; i < numInputs; i++)
  weight[i] += lRate * grad[inputSet] * input[inputSet][i];
}

bool complete()
{
 // Check if the network output is sufficiently close to the goal
 for(int i = 0; i < 4; i++)
 {
  if(fabs(GOAL[i] - network[0][NUMBER_OF_LAYERS - 1].getOutput(i)) > 0.01) 
   return false;
 }
 return true;
}

int main()
{
 bool result = false;
 // Initialize the network
 for(int i = 0; i < NUMBER_OF_LAYERS; i++)
 {
  for(int j = 0; j <= NEURONS_PER_LAYER; j++)
   network[j][i].initialize(i, NEURONS_PER_LAYER, 0.25);
 }
 cout << "\n";
 // Train the network
 cout << "Goal: [ ";
 for(int j = 0; j < 4; j++)
  cout << GOAL[j] << " ";
 cout << "]\nLayer " << NUMBER_OF_LAYERS - 1 << " output:\n";
 for(int i = 0; i < MAX_CYCLES; i++)
 {
  // Forward pass: layer 0 -> (NUMBER_OF_LAYERS - 1)
  for(int j = 0; j < 4; j++)
  {
   for(int l = 0; l < NUMBER_OF_LAYERS; l++)
   {
    for(int m = 0; m <= NEURONS_PER_LAYER; m++)
     network[m][l].forward(j);
   }
  }
  // Reverse pass: layer (NUMBER_OF_LAYERS - 1) -> 0
  for(int j = 0; j < 4; j++)
  {
   for(int l = NUMBER_OF_LAYERS - 1; l >= 0; l--)
   {
    for(int m = NEURONS_PER_LAYER; m >= 0; m--)
     network[m][l].updateDeltas(j);
   }
  }
  for(int j = 0; j < 4; j++)
  {
   for(int l = NUMBER_OF_LAYERS - 1; l >= 0; l--)
   {
    for(int m = NEURONS_PER_LAYER; m >= 0; m--)
     network[m][l].updateWeights(j);
   }
  }
  // Display the network's output, and check if it is complete
  cout << "[ ";
  for(int j = 0; j < 4; j++)
   cout << network[0][NUMBER_OF_LAYERS - 1].getOutput(j) << " ";
  result = complete();
  cout << "]\n";
  if(result)
  {
   cout << "\n**** Training successful after " << i << " cycles! ****\n";
   i = MAX_CYCLES;
  }
 }
 if(!result)
  cout << "\n**** Training failed. ****\n";
 cout << "Goal: [ ";
 for(int j = 0; j < 4; j++)
  cout << GOAL[j] << " ";
 cout << "]\n";
 cout << "Final network weights:\n";
 for(int j = 0; j < NUMBER_OF_LAYERS; j++)
 {
  for(int k = 0; k < NEURONS_PER_LAYER; k++)
  {
   cout << "Layer " << j << ": [ ";
   for(int l = 0; l < network[k][j].getNumInputs(); l++)
    cout << network[k][j].getWeight(l) << " ";
   cout << "]\n";
  }
 }
 cout << "\n";
 return 0;
}

So all these roads go to Damascus...

Having done a little digging, I have mixed feelings about Sarah Palin. To the extent she's got a libertarian streak, I like and respect her. She's a bit of a rebel, which I also respect, particularly from the bunch of hooligans that dominate the Republican party in Alaska. But the Christian dominionist stuff really, really turns me off, particularly the wildly irresponsible talk about the Iraq war being a mission from God:

http://www.youtube.com/watch?v=QG1vPYbRB7k
http://www.youtube.com/watch?v=k84m2orSOaM
http://www.politico.com/news/stories/0908/13098.html

Get your heart right with God, guys, or we'll never get that pipeline up! You know, John McCain used to be famous for speaking out against this stuff. It's cringe-inducing, watching a guy I used to admire (I would have probably voted for him in 2000, in fact) abandon his principles for power. In McCain's defense, I doubt he knew about this prior to selecting her. Although that's not much of a defense, when you consider what it implies about his decision-making process.

Political considerations aside, how does this stack up against my reservations about the Obama/Biden ticket's collectivist-minded economic policies? Obama's stated support for a Windfall Profits Tax on the oil industry reeks of collectivism, but it's gimmicky and probably just a lame pander when you get right down to it; things like his support for the Fair Pay Act of 2007 are much more ominous, in my opinion. It's hard to say whether this is more or less alarming than a candidate who believes she's on a mission from God. It's all enough to make me seriously think about voting for Bob Barr, to be perfectly honest. I'm not sure how sincere his little road-to-Damascus moment regarding the Drug War was, either, but you'd have to be a damn fool to sell your soul for the LP nomination, so him I'll give the benefit of the doubt. But John McCain's at the top of the ticket, not Sarah Palin, so I'll probably be voting for Obama this year. (Although, since I live in California, it matters very little who I vote for in any case.)