Ok.  So here's the finished product.

The most permanent thing I did was to use velcro that had adhesives and put a few inches onto my kayak.  Other than that, no holes, nothing permanent.

It is ROCK solid.  I think it's going to work,  Now its up to the operator.

I used hooks to hold the two sides of the board down.  The board was too thin so I drilled a hole in the top of a PVC pipe cap and added a washer and wingnut.  The front is just wedged in and held in place by the bungie cords.