Transferring MiniDV Tapes to Linux

Downloading miniDV tapes from my Sony DCR-TRV22 camcorder to my Fedora 32 Linux system with a Thunderbolt 3 port was easy, using dvgrab and a couple of Apple converters to go from FireWire to Thunderbolt 3.

Many years ago I transferred all my VHS home videos to disk through the somewhat painful process of first recording them onto DVDs using a DVD recorder, then ripping the DVDs on my computer. My next video transfer project was to transfer my more recent home videos from miniDV to disk. There was always other work to do, and transferring the tapes was never a critical task, so it was easy to put off. I was thinking it probably would be time consuming but not too difficult, since my Linux computer had an IEEE 1394 (FireWire) port, so I wasn't too worried about it.

When the lockdown started earlier this year, that presented a good opportunity for me to start my tape transfer project. I grabbed my miniDV camcorder and my box of tapes, then went to get a cable to connect the camcorder to my computer. It was only then that I remembered that I upgraded to a new computer at the beginning of this year and gave away the old computer. The old computer, from 2010, had the IEEE 1394 port, the but new one did not. Oops! I waited a bit too long for this supposedly easy job.

My new computer has a ton of ports of various flavors, so it seemed possible that it might still work, if I could get the right cables and converters. After some digging, it looked like it should be possible to use the USBC Thunderbolt port on my new computer. But I couldn't find much support for whether it would work when run through converters on a current version of Linux. The required converters are pretty expensive, but I decided to take a chance and buy them.

My Sony DCR-TRV22 camcorder has a 4-pin FireWire 400 jack, and I had FireWire 400 to 800 cable. I purchased an Apple Thunderbolt to FireWire Adapter for $29 and an Apple Thunderbolt 3 (USB-C) to Thunderbolt 2 Adapter for $49, and for good measure I also purchased a FireWire 400 to 800 Adapter for $10 (in case I had to use a different cable), which I ended up not using. I connected the cable to the camcorder, connected the other end of the cable to the FireWire to Thunderbolt adapter, plugged the FireWire to Thunderbolt adapter into the Thunderbolt 2 to Thunderbolt 3 adapter, and plugged the Thunderbolt 2 to Thunderbolt 3 adapter into the USBC Thunderbolt 3 port on my computer. Then I ran dvgrab, which I had installed earlier. And... it did not see the camera. Rats.

# lsmod | grep -i fire
  (nothing)
# lspci | grep -i fire
  (nothing)

Fortunately, it turned out to be an easy fix. I was able to determine that the Thunderbolt to FireWire adapter was visible by looking in /sys/bus/thunderbolt:

# cat /sys/bus/thunderbolt/devices/0-3/device_name
  Thunderbolt to FireWire Adapter

I found the solution in an Ubuntu bug report: the Thunderbolt device had to be authorized. (Note that your device number might be different.)

# cat /sys/bus/thunderbolt/devices/0-3/authorized
  0
# echo 1 > /sys/bus/thunderbolt/devices/0-3/authorized
# lspci | grep -i fire
  40:00.0 FireWire (IEEE 1394): LSI Corporation FW643 [TrueFire] PCIe 1394b Controller (rev 08)
# lsmod | grep -i fire
  firewire_ohci          45056  0
  firewire_core          81920  1 firewire_ohci
  crc_itu_t              16384  1 firewire_core

At this point I was able to insert a tape into the camcorder and test it:

$ dvgrab foo-

This created the file foo-001.dv. By installing the mediainfo program, I was able to see the datestamp of the recording:

$ mediainfo foo-001.dv | grep date
  Recorded date                            : 2015-12-25 10:32:28.000

The actual command I used to download the tape is:

$ dvgrab --autosplit --timestamp --size 0 --rewind --showstatus dv-

At this point I could just put a tape in the camcorder, rewind it, run the above dvgrab command, come back an hour or so later when it was done, then put in the next tape and repeat. It took a long time to get through all my miniDV tapes, but not much work.

Go Composition vs Inheritance

Go does not support inheritance, but sometimes using embedded structs can look a little like inheritance. I explore that feature to see how it differs.

Contents

• Introduction
• Base class
• Subclass
• Main and test
• Overriding
• Downcall
• Method promotion
• Solution
• Conclusion

Introduction

In lieu of inheritance, the Go language encourages composition by allowing one struct to be embedded in another struct in a way that allows calling methods defined on the embedded struct as if they are defined on the containing struct.

Note: In this post I occasionally use object-oriented terminology such as base class, subclass, and override. Please remember that Go does not support these concepts; I am using those terms here to show how thinking that way with Go can lead to problems.

For the examples that follow, I assume we are building a graphical editor that allows manipulating visual objects on the screen. We want to be able to draw those objects, and we want to be able to transform them with operations such as rotate, so we define an interface with those methods:

Note: For convenience, the final collected code used in this post is available on play.golang.org.

type shape interface {
  draw()
  rotate(radians float64)
  // translate and scale omitted for simplicity
}

We write a function that will draw all our shapes:

func drawShapes(shapes []shape) {
  for _, s := range shapes {
    s.draw()
  }
}

Base class

We define our "base class", called polygon, where we implement a draw method that we can invoke from our "subclasses":

type polygon struct {
  sides int
  angle float64
}

func (p *polygon) draw() {
  fmt.Printf("draw polygon with sides=%d\n", p.sides)
  vertexDelta := 2*math.Pi / float64(p.sides)
  vertexAngle := p.angle
  x0 := math.Cos(vertexAngle)
  y0 := math.Sin(vertexAngle)
  for i := 0; i < p.sides; i++ {
    // Draw one side within unit circle, offset by p.angle.
    vertexAngle += vertexDelta
    x1 := math.Cos(vertexAngle)
    y1 := math.Sin(vertexAngle)
    fmt.Printf("draw from (%v, %v) to (%v, %v)\n", x0, y0, x1, y1)
    x0 = x1
    y0 = y1
  }
}

func (p* polygon) rotate(radians float64) {
  p.angle += radians
}

Subclass

We define a couple of "subclasses", triangle and square, that "extend" our "base class", along with functions to create instances of those types:

type triangle struct {
  polygon
}

type square struct {
  polygon
}

func createTriangle() *triangle {
  return &triangle{
    polygon {
      sides: 3,
    },
  }
}

func createSquare() *square {
  return &square{
    polygon {
      sides: 4,
    },
  }
}

Main and test

Finally, we write a couple of test functions to create a list of shapes and draw them, and a one-line main function that calls our test function.

package main

import (
  "fmt"
  "math"
)

func createTestShapes() []shape {
  shapes := make([]shape, 0)
  shapes = append(shapes, createTriangle())
  shapes = append(shapes, createSquare())
  return shapes
}

func testDrawShapes() {
  drawShapes(createTestShapes())
}
func main() { testDrawShapes() }

When we run this program, it produces the expected output:

draw polygon with sides=3
draw from (1.000, 0.000) to (-0.500, 0.866)
draw from (-0.500, 0.866) to (-0.500, -0.866)
draw from (-0.500, -0.866) to (1.000, -0.000)
draw polygon with sides=4
draw from (1.000, 0.000) to (0.000, 1.000)
draw from (0.000, 1.000) to (-1.000, 0.000)
draw from (-1.000, 0.000) to (-0.000, -1.000)
draw from (-0.000, -1.000) to (1.000, -0.000)

Note that we have not defined any methods on the triangle and square types, yet the compiler accepts them as implementing shape, as seen by the fact that we can store them in a slice of shape and we can invoke draw on them. Because we embedded polygon in triangle and square, without giving them field names, Go has promoted all of the methods in polygon into the namespaces of triangle and square, allowing draw to be called directly on an instance of type triangle or square.

So far, relying on an object-oriented mental model has not caused us problems. Let's keep going and see when it does.

Overriding

We add a typeName method to our shape interface and our "base class", polygon, and we "override" that method in our "subclasses", triangle and square:

type shape interface {
  draw()
  rotate(radians float64)
  // translate and scale omitted for simplicity
  typeName() string
}

func (p *polygon) typeName() string {
  return "polygon"
}

func (p *triangle) typeName() string {
  return "triangle"
}

func (p *square) typeName() string {
  return "square"
}

We can test our typeName methods by pointing our main to a different test function:

func printShapeNames(shapes []shape) {
  for _, s := range shapes {
    fmt.Println(s.typeName())
  }
}

func testShapeNames() {
  printShapeNames(createTestShapes())
}
func main() { testShapeNames() }

This outputs:

triangle
square

No problems yet.

Downcall

Let's add a method to our interface and "base class" that invokes the method that we are overriding, and a new test function to call it. This is sometimes referred to as a downcall, in that a superclass calls into the overriding method of a subclass that is below it in the class hierarchy.

type shape interface {
  draw()
  rotate(radians float64)
  // translate and scale omitted for simplicity
  typeName() string
  nameAndSides() string
}

func (p *polygon) nameAndSides() string {
  return fmt.Sprintf("%s (%d)", p.typeName(), p.sides)
}

func printShapeNamesAndSides(shapes []shape) {
  for _, s := range shapes {
    fmt.Println(s.nameAndSides())
  }
}

func testShapeNamesAndSides() {
  printShapeNamesAndSides(createTestShapes())
}
func main() { testShapeNamesAndSides() }

This outputs:

polygon (3)
polygon (4)

Well, that doesn't look right. We wanted it to print triangle and square instead of polygon both times. Thinking of this as inheritance has led us astray.

Method promotion

So, what happened here? Why did printShapeNames work, but printShapeNamesAndSides did not? Let's dig into that.

The return value of createShapes is []shape, which is a slice of objects that implement the shape interface. Since the triangle and square types implement that interface, we can store instances of those types in that slice. But how is it that those types implement that interface when we didn't write those methods for those types? The answer is method promotion.

When we embed one type inside another without giving the internal type a field name, Go automatically promotes all unambiguous names from the embedded type to the containing type. Effectively, for each method in the embedded type whose name does not conflict with a method in the containing type or in any other embedded type within that container, Go creates a method on the containing type that turns around and calls that method on the embedded type. For example, when we embed polygon in triangle the compiler effectively creates this code:

func (t *triangle) typeName() string {
  return t.polygon.typeName()
}

If the embedded type satisfies an interface, and there are no ambiguous method names, this promotion of all the methods of the embedded type makes the containing type also satisfy that interface. Let's explore this method promotion behavior. We create another struct type called thing that has a typeName method, embed it along with our previously defined polygon, which also has a typeName method, in a new type polygonThing, then try to assign an instance of that to a variable of type shape.

type thing struct{}
func (t *thing) typeName() string { return "thing" }

type polygonThing struct {
  polygon
  thing
}

func testPolygonThing() {
  p := &polygonThing{}
  p.draw()
  fmt.Println(p.typeName())
  var s shape = p
  fmt.Println(s.typeName())
}
func main() { testPolygonThing() }

When we compile this, we get these errors:

./comp.go:130:16: ambiguous selector p.typeName
./comp.go:131:7: polygonThing.typeName is ambiguous
./comp.go:131:7: cannot use p (type *polygonThing) as type shape in assignment:
        *polygonThing does not implement shape (missing typeName method)

where line 131 is the line where we are assigning to s.

From this error we can see that Go did not promote the typeName method from either of the embedded structs into polygonThing. But there was no error message about the call to draw, so it did promote that method from polygon, since it is not ambiguous.

If we comment out the embedded thing line from the definition of polygonThing, the code compiles. If, instead, we comment out the embedded polygon line, we get different errors:

./comp.go:129:4: p.draw undefined (type *polygonThing has no field or method draw)
./comp.go:131:7: cannot use p (type *polygonThing) as type shape in assignment:
        *polygonThing does not implement shape (missing draw method)

If we want to keep both embedded structs in our composite struct, there are a couple of ways we can resolve the ambiguity of typeName appearing in both embedded structs. The simplest is to assign a name to one of the embedded structs, converting it to a regular field. Instead of writing thing in the definition of polygonThing, we can write t thing. Go then does not attempt to promote the methods from thing into polygonThing, and the promotion of typeName from polygon into polygonThing is no longer ambiguous, so it succeeds.

Another possibility is to resolve the ambiguity by defining a typeName method directly on polygonThing. In this case, Go does not attempt to promote typeName from either of the embedded structs. We can call a method in an embedded struct by referring to that embedded struct as if it were a named field.

func (t *polygonThing) typeName() string {
  return t.polygon.typeName()+"Thing"
}

With this definition, the program compiles and runs, outputting

draw polygon with sides=0
polygonThing
polygonThing

Solution

Now that we understand how embedded structs work in Go, let's go back and reconsider what happened with our printShapeNamesAndSides function.

Assume one of the elements in our slice of shape is an instance of triangle. We call nameAndSides with that triangle as the receiver. Since we did not define nameAndSides on triangle, that calls the promoted version of that method. That promoted method turns around and calls nameAndSides on the embedded polygon, passing the embedded polygon as the receiver. In polygon.nameAndSides, it calls p.typeName, but p here is the receiver of the nameAndSides method, which is the polygon, not the triangle. So the call from nameAndSides to typeName call's the typeName method on polygon rather than on triangle.

With this understanding, let's update our code to make "overriding" work. The difference between the behavior we are seeing and what we would expect from a system with inheritance and overriding is that here our "base class" does not, by default, make calls to methods of the "subclass". It can't because the method in the "base class" has no reference to the type of the containing object. In order to implement a call to method in an instance of a "subclass" from polygon.nameAndSides, we need a reference to that instance, such as a triangle. We will do this by explicitly passing our shape as an argument, then calling the typeName method on that shape rather than on the receiver. By calling a method on a passed-in argument rather than the receiver, it is clear, when looking at that method in the "base class", that the call may be going to a different type of object than polygon.

type shape interface {
  ...
  nameAndSides(s shape) string
}

func (p *polygon) nameAndSides(s shape) string {
  return fmt.Sprintf("%s (%d)", s.typeName(), p.sides)
}

func printShapeNamesAndSides(shapes []shape) {
  for _, s := range shapes {
    fmt.Println(s.nameAndSides(s))
  }
}

With these changes, we get the expected output:

triangle (3)
square (4)

Conclusion

The way Go promotes methods of embedded structs makes it have some of the characteristics of inheritance as defined in object-oriented programming. In particular, it allows for methods to be automatically promoted to the containing struct, and thus for interfaces to be automatically promoted to the containing struct. One key difference is that, when you override one of those promoted methods in the containing struct, the code in the embedded class does not automatically call the overridden method in the containing class, as happens in some object-oriented languages such as Java.

You may have heard of the fragile base class problem. A related issue, that can arise when there are downcalls from a superclass to an overridden method in a subclass, similar to the example here where I "overrode" the typeName method, might be termed the fragile subclass problem. If you are interested into digging into that, you can read Safely Creating Correct Subclasses without Seeing Superclass Code, a paper from OOPSLA 2000 that examines that issue. See section 4. The designers of Go chose not to implement inheritance, but instead to favor composition. Although some Go constructs can look a little like inheritance, it's better to start thinking about designing in Go using composition rather than trying to bend Go to do something like inheritance.

A Future Telescope

This post describes an idea for a telescope that can see where heavenly objects will be in the future. This may sound crazy, like something out of a science-fiction story, but I believe it is based on solid theory. Unless, or course, I have misinterpreted something. Read on if you enjoy considering surprising extrapolations of theory.

Contents

• Collective Electrodynamics
• Interpreting the Theory
• The Big Idea
• The Details
• An Invitation

Collective Electrodynamics

Carver Mead's book Collective Electrodynamics, first published in 2002, puts forth a theory of electrodynamics based on four-vectors. As with many other low-level aspects of physics, this theory is time-symmetric, making no claims about how to distinguish between the past and the future.

I found Carver's theory and his exposition of it to be elegant and convincing. Even if you don't agree with my interpretation and conclusions in this post, I recommend you read this book if you are generally interested in physics.

Carver's description of the process of photon emission and absorption includes a few comments noting that a photon will not be emitted without a destination that will absorb the photon at some point in the future, because the emitter and absorber are a coupled pair forming a single resonator.

• In section 4.8: "Any energy leaving one resonator is transferred to some other resonator, somewhere in the universe."
• In section 4.12: "The spectral density of distant resonators acting as absorbers is, of necessity, identical to that of the resonators producing the local random field, because they are the same resonators."
• In the Epilogue: "It is by now a common experimental fact that an atom, if sufficiently isolated from the rest of the universe, can stay in an excited state for an arbitrarily long period. ... The mechanism for initiating an atomic transition is not present in the isolated atom; it is the direct result of coupling with the rest of the universe."

Part 5 describes how two atoms couple electromagnetically as resonators.

Interpreting the Theory

As a thought experiment, if we were out in space in some part of the universe in which there were no matter in one direction, we would not be able to shine a flashlight in that direction because there would be nothing to absorb the photons, therefore they would not be emitted. If we were able to measure all of the other energy going into or out of the flashlight, we would be able to notice that energy leaves the flashlight when we point it towards other things, but not when we point it towards truly empty space.

Coming back to our current location in the universe, there is a finite amount of matter between us and the Hubble sphere. Consider a line segment from our location to a point on the Hubble sphere. If there are no atoms on the intersection of said line segment and our future light cone, then it should not be possible to emit a photon in that direction. More restricted, if there are no atoms in that intersection that are capable of absorbing a photon of the frequency our source atom is attempting to emit, then we will not be able to emit said photon in that direction.

The Big Idea

Assume, then, that we have a highly directional monochromatic light source that we can point accurately, and that we can accurately know how much light we are emitting based on energy input measurements. What would happen if we were to provide that light with a suitable input power signal, then scan the sky? If there are any differences in the density of atoms in different directions that are capable of absorbing photons of the frequency we are sending, would we be able to produce a map of the sky showing those differences? Would there be any anisotropism, as there is for the background radiation?

Given how much matter there is in the universe, I suspect it would be hard to find one of those line segments out to the Hubble sphere without a single atom capable of absorbing one of our photons, but perhaps if we are trying to send out a great many photons, there will be enough of a statistical variation to measure.

The thing that I find fascinating about this is that, if it did in fact work, we would be "seeing the future", because whatever map we produced would be a function of where the absorbing atoms are going to be when the light we emit reaches them. For planets in our solar system that would be minutes or hours in the future, but for distant nebulae that could be millions or billions of years from now.

The Details

The devil is in the details. Even if, in principle, the theory supports this conclusion, would it be possible to build such a device?

In addition to the statements of theory, I make two assumptions above:

1. We can accurately point our light source, such that we can perform a raster scan on a portion of the sky.
2. We can determine how much light energy is leaving our light source by measuring the input energy to that source.

The first assumption seems straightforward: the optics involved in sending out a beam of light to a small portion of the sky should be the same as receiving light from a small portion of sky, which we do on a regular basis to form images of space. But I am not an astronomer, so I may be missing something. For example, I know that some modern telescopes use a guide laser shining up through the atmosphere to allow for dynamic adjustments to the mirrors to compensate for atmospheric distortion. Would this also work when sending out a signal beam alongside the reference beam? I don't know why not, but, as mentioned, this is not my area of expertise.

I think the second assumption may require more effort to solve. The typical advice for powering a laser is to use a current source in order to get a stable output. For my experiment, however, I specifically don't want a stable source. Instead, I want a source that can output more or less light based on how much the space into which it is shining can accept.

Since I can't directly measure the light output, I also need a light source where I can accurately judge how much light is being output by measuring the input power. This means I need to know the power transfer characteristics of the light source. How much of the input power is transformed into light, and how much into heat or other forms of energy? Is that relationship constant over time, or might it vary such that at one point in time I get x% of the input turning into heat, and moments later I get 2x% turning into heat? Alas, I am not a solid-state physicist (assuming my light source is a solid-state laser), so I don't know the answers to these questions.

An Invitation

So, what do you think? Is there a fatal flaw to my understanding of the theory? A fundamental reason why it would not be possible to build such a "future telescope"? A technical limitation making it not currently possible?

I have talked to a few people about this idea, and the ones who I know have a good understanding of Carver's theory have said that, in principle, they don't see anything wrong with my reasoning.

AsI mentioned above, I'm not an astronomer or solid-state physicist, so I don't have the background to take this concept to the practical stage. But perhaps someone else does.

This seems like it would be a very exciting thing if it worked, but I think it would require a significant investment of time and access to some expensive equipment to take the next step. Would anyone like to give it a try? If you do, I'd love to hear about it.

Wormhole Musings

I have questions about how wormhole portals in science fiction stories work.

Recently I started reading another science fiction novel where wormholes allow instantaneous travel between distant points. In books that use this mechanism, the author typically explores how the ability to travel easily and quickly between the stars shapes the course of history.

But I always get hung up thinking about all the other ways in which a portal might possibly be used, for good or evil, in ways much less grand but potentially more disruptive than distant travel. Of course, since the use of wormholes in these books does not rely on our currently generally accepted science, these questions do not have well-defined answers. That's why I muse.

In this article, I ask some questions about how some of our currently accepted principles of physics apply (or don't apply) to wormholes, and ponder the ways in which one might use (or misuse) a wormhole based on the answer to those questions.

Caveat lector: If you want to keep reading wormhole stories without being distracted by questions like these, you might want to stop reading now. Because once you read these questions, you won't be able to unread them.

My Questions

• How big and expensive is the equipment required to create and maintain a wormhole?
• How much energy is required to create and maintain a wormhole?
• What shape is a wormhole?
• Can I make a wormhole as large or as small as I want?
• How do you control the location of the wormhole portals?
• Is energy conserved when traversing a wormhole?
• Is momentum conserved when traversing a wormhole?
• How do physical forces propagate through a wormhole?
• What is the geometry of the wormhole connection?
• In what reference frame is traversal of the wormhole instantaneous?

• Potential Answers

How big and expensive is the equipment required to create and maintain a wormhole?

Mainly what I want to know for this question is whether the equipment is small and inexpensive enough that an individual can own one. If they are within the reach of many people, that makes it much more likely that there will be some people who will use it for unexpected purposes.

I once read a story in which someone had invented a personal flying belt that anyone could get for five dollars. With such easy personal mobility, border control suddenly became much more difficult, which of course led to some interesting problems. If anyone could buy and control a wormhole for five dollars, that would be a very different situation than if there were only a few wormholes controlled by a few rich and powerful entities.

How much energy is required to create and maintain a wormhole?

Although science fiction wormholes don't rely on any currently known physics, my feeling is that any scientifically plausible mechanism for a wormhole would require a prohibitive amount of energy to use. And I mean the word prohibitive literally: the amount of energy required would be so high, it would effectively prohibit the possibility of using a wormhole.

Since that doesn't make for good science fiction stories, we have to assume that the energy requirement is modest enough that we are able to produce and use wormholes. The question then becomes, how much energy is required? This question is related to the earlier question about cost, in that if a wormhole requires a relatively large amount of energy to operate, that could restrict its operation to a small number of controlling entities. Whereas if I can run it with a D-cell battery, there would be many more interesting things I could do with it.

It may not matter how much energy it requires to operate a wormhole, because, as discussed in some of my comments below, it seems likely that once you have a wormhole you could get as much free energy as you want.

What shape is a wormhole portal?

In most stories, wormholes portals are portrayed as circular areas that you step through, much like the entrance to a common tunnel. This is very convenient for imagining things like train lines that run through wormholes, and for thinking about the equipment that might be required to hold open a wormhole portal. That equipment is sometimes described as a torus with massive structures around it.

I think it is more likely that a wormhole would be spherical. You could enter it from any direction, and you would exit in a direction based on the direction you entered. This is a bit harder to visualize, which may be one reason it is not often described this way.

If a wormhole portal is a sphere, how does that impact the equipment required to maintain it? It would be tough to have equipment symmetrically on all sides and still have something that allows easy access. But maybe it doesn't all have to be completely symmetrical, so you can leave a few holes to let the trains get through the equipment so they can enter the portal.

Can I make a wormhole as large or as small as I want?

In most stories, wormholes are of a size that makes them convenient to step through, or drive a car or train through. Is this an essential feature of wormholes, or is it just that that happens to be the most convenient size? Could we make them any size if we wanted to? Perhaps big wormholes would be harder, but I would think smaller wormholes would actually be easier to make. And I can think of lots of interesting uses for small wormholes, depending on the answers to the other questions.

One example of a good use for a tiny wormhole would be to shine a laser through it and have a high capacity communication channel.

How do you control the location of the wormhole portals?

Some stories postulate that maintaining a wormhole portal requires physical equipment at both ends. In this case, the question of how to control the location of the portal is clear: you have to move the equipment to move the portal.

In other stories, the two ends of the wormhole are created at one location, after which one end can be moved to another location. In considering the geometry of wormholes, I would guess that it is possible to move one end of a wormhole through another wormhole, but perhaps only if the wormhole being transported is sufficiently smaller than the one it is being moved through.

If equipment is required at both ends of the wormhole, establishing a wormhole from A to B requires first traveling from A to B through normal space to deliver the necessary equipment, or possibly from C to B if the two ends of the wormhole don't need to be created in one place. This constrains the expansion of an interstellar civilization to the speed of light, which is annoyingly slow to some authors.

The more interesting case, as postulated in some stories, is that you can project the other end of the wormhole to a desired location without first having to get there some other way. This is, of course, a much-preferred mechanism if you want to quickly expand your network of gates, since who wants to wait many years while the slowship takes your gate to the next star? But what could we do if we could project the other end of our portal to anywhere we wanted in space?

If I can project tiny wormholes, I could do cut-less surgery. Mining would be much cheaper, as I could just project a wormhole down to where the ore is without having to tunnel or strip-mine down to it. I could make a great vacuum pump by putting one end out in space.

At a more banal level, I could eat as much as I want and not gain weight. I just need to project a tiny wormhole into my stomach and remove the food I just ate before my body digests it. I get all the pleasure of eating without suffering the problems of obesity.

I read one story in which a little wormhole was located on the bottom of a drinking glass, with the other end at the bottom of a vat of beer, wine, or whatever drink was selected. Each time the glass was set down, the wormhole would open to fill the glass, then close once the glass was full.

If I put on my black hat, the most obvious nefarious deed is, I project the other end of my wormhole into a bank vault and walk off with the cash. Or into a collection of classified documents and walk off with the secret plans. Or into my enemy's bedroom and kidnap him or kill him. I really only need to project a tiny wormhole, big enough for a bullet, to do a dastardly deed. Or so small it's only big enough for a packet of viruses that I inject into his bloodstream without him even knowing it.

If we can project one end of our wormhole to any desired location in space, perhaps we could project both ends. This would allow us to establish a wormhole between any two points anywhere in space, without having to have equipment at either end. This could actually be an interesting premise for a story, as it would allow for the case where there is a single wormhole-generating facility that creates all of the wormholes used throughout the civilization. That facility would presumably be controlled by some now-very-powerful entity, and would be both heavily secured and heavily attacked, so there are lots of opportunities for story lines.

The ability to create a wormhole between any two other points in space also opens up lots of additional opportunities for mischief. One could create a pretty effective weapon of mass destruction by creating a wormhole with one end in the middle of the sun and the other end where you want the destruction. Or put one end in the middle of a magma reservoir, or deep in the ocean, depending on the type of destruction desired. Or put one end in space to suck everything into the vacuum.

On the positive side, one could create a really nice package delivery system. Open a wormhole between the package source and destination, drop the package in for instant delivery, and close the wormhole.

Assuming we have the ability to create a wormhole portal anywhere in space, there is still the question of how we figure out where it gets created. Do we have to use trial and error to place the wormhole in just the right place? If we are trying to create a wormhole portal in a distant location, do we have to worry about the precision of our equipment, in the same way that launching a spaceship to land on Mars requires more precise equipment than launching one to land on the moon? Can we create the remote wormhole portal and then move it around at will, and if so, can we move it faster than the speed of light?

Is energy conserved when traversing a wormhole?

In most wormhole stories, one can step through a wormhole to get from one end to the other with no more effort than walking across the room. There is no explicit discussion of conservation of energy, and my assumption is that the authors don't worry about it because that detail doesn't advance the story. But I worry about it.

If I open a wormhole between Earth and its moon, there is a pretty big difference in the gravitational potential energy between those two points. When I want to put something in the wormhole portal on Earth and have it come out on the moon, do I need to supply the difference in energy between those two points? That would mean supplying a whole lot of energy to move in that direction. Conversely, if I step through the wormhole from the moon back to the Earth, what happens to all that gravitational potential energy?

If I can move from one end of a wormhole to the other end without having to supply that extra energy, then I can get free energy. Here's one way: go find a big dam with a hydro generating plant and install a wormhole with the entrance portal under the water at the bottom of the dam, just past the outflow of the generator, and with the exit portal just above the surface of the lake at the top of the dam. Since the entrance portal is underwater and the exit is above, water flows into the entrance portal and comes out at the exit portal. Thus the lake is ever refilled and our hydroelectric generators can keep running.

Maybe the wormhole technology works like a battery with regenerative braking on electric cars: it supplies the energy needed when traveling in one direction, and absorbs the excess energy when traveling in the other direction.

Is momentum conserved when traversing a wormhole?

If I am in New York City, the Earth's rotation is moving me at about 700 miles per hour relative to the center of the Earth. At the same time, Sydney is also moving at about 700 miles per hour, but in roughly the opposite direction, as it is almost on the opposite side of the Earth. If I open a wormhole between New York City and Sydney, and I step through, what happens to that 1400 miles per hour difference? Do I splat into the nearest wall at supersonic speed, or do I casually step through and continue walking to my destination?

If momentum is conserved, then I would be moving at a high speed relative to the exit point of the wormhole. If I put the appropriate mechanical devices next to the wormhole exit, I could send through a rock, catch it moving at 1400 miles per hour, and convert that kinetic energy to electricity. Then I could toss the rock back and do the same thing on the other side. Free energy.

The question of conservation of momentum is subtler than it first appears. If I want to conserve momentum, I come out of the wormhole in Sydney with that supersonic velocity relative to the city. But what does that mean for the angular momentum of the system? If I just moved that mass over to a new location and nothing else changed, then I have changed the angular momentum of the system. If the whole earth moves a tiny bit in the other direction, to keep the same center of mass, that could take care of that issue, but why should the whole Earth move when I use a wormhole? Would that happen if I were in an airplane? In a spaceship in low orbit? In a spaceship in high orbit? In a spaceship at the orbit of the moon, or beyond?

As with conservation of energy, perhaps the wormhole portals absorb or supply momentum as needed, transferring it to the surrounding masses. This could mean that wormhole portals would most effectively be placed on large masses such that they had a reservoir of momentum to transfer to or from. The larger the masses that were transferred through a wormhole, and the larger the relative velocity of the portals, the more momentum would have to be transferred, and the larger the attached mass would have to be.

How do physical forces propagate through a wormhole?

In every wormhole story I have read, light traverses a wormhole with no problems. I assume that means all forms of electromagnetic radiation traverse a wormhole equally easily. This presents another opportunity for a good energy source: put a wormhole portal in close orbit around the sun, then put the other wormhole portal on Earth. Stream that high-intensity light through and use it to drive solar cells for direct production of electricity, or as a heat source for standard steam turbines. If no equipment is required at the solar end of the wormhole, you're all set. If equipment is required, you might have to build some kind of refrigerator that brings that heat back to Earth and keeps the equipment cool.

How about gravity? How does that propagate through a wormhole? Most wormhole stories I have read describe travelers stepping through a wormhole and experiencing a discontinuity in the gravity field, meaning gravity is not propagating through the wormhole. This seems odd to me. Why would light propagate through a wormhole but not gravity?

The intensity of light from a point source drops off proportionally to the distance squared, which makes sense because the light is spreading out at that rate, and a fixed-size object intercepting the light will thus get less of it when it is further away. Because of this behavior, it makes sense to me that the amount of light that would come through a wormhole would be proportional to its size. If the wormhole is very small, only a small amount of light would come through.

Gravity also drops off proportionally to the distance squared, but not quite for the same reason. Given a particular mass, the gravitational force on that mass is independent of whether it is small and dense, or larger and less dense. The amount of area covered by the mass is not important, only its mass and its distance from another mass. If there is a tiny wormhole and I can measure a distance through that wormhole from my object to a large mass, wouldn't that mean the gravitational force is proportional to the square of that distance?

If gravity does propagate through a wormhole, perhaps I could make a null-gravity region by creating a pair of wormhole portals, then putting each one slightly above the surface of the Earth and upside down from each other. If you were to stand under one portal and look up, you would see the Earth above you. You have one Earth gravity below you and one above, so they cancel out and you have no gravity. A nice tourist attraction. Then again, the two Earths would also be exerting a gravitational pull on each other, so whatever is holding up each wormhole portal might be carrying the weight of the world.

On the other hand, given that General Relativity says that mass causes curvature of space, and thus gravity, and wormholes are usually described as some way of warping space, that seems to imply that being able to control wormholes means being able to control the curvature of space and thus being able to control gravity. So perhaps based on that we can choose how we want gravity to propagate through wormholes for our stories.

If you can turn wormholes on and off at will, you might be able to use this effect to get some free energy. You turn on a wormhole, have it pull up a weight, then turn it off, let the weight fall, and use that to generate energy.

What is the geometry of the wormhole connection?

A wormhole is usually described as a connection that goes through a higher dimension than the three dimensions in which we live. Those higher dimensions may present degrees of freedom that can lead to some curious and unpleasant results. Let me try to explain with a flatland analogy.

If I live in a two dimensional space, I can create a wormhole by folding that sheet of space until two points meet, then punching out a circle around those two points, and sewing those two circles together. This is topologically equivalent to attaching a hose that stretches up from a circle around one of those points and comes down at a circle around the other, with the assumption that the hose represents no distance (or a very short distance). A 2D creature could move from regular space onto the surface of that hose (assuming the hose diameter is much larger than the creature), then to regular space on the other end, then return to its original location via regular space, and all is well.

Now consider what happens if I take that same hose, but instead of going up from the first point and down at the second, I go up from the first point, then go around to the under side of the plane (which I can do without going through the plane if I have yet another dimension) and come up from the bottom side of the plane to meet the second point. Consider again what happens to that 2D creature who travels into the wormhole, out the other end, and returns to its starting point in normal 2D space. The result is that it comes back inverted. What was left is now right, and vice-versa.

I once read an old science fiction story in which there was a place deep within the Amazon where, if you navigated a certain course, it would reverse everything left to right. An enterprising businessman heard this and figured he could more efficiently make shoes by manufacturing only left shoes, then shipping half of them around this circuit, so he went exploring to find it. After going around the course, he looked at his sample left shoes, but they were all still left. Frustrated, he threw them all away, destroyed the worthless maps, and returned to civilization - only to discover that in fact the trick had worked, but he had not recognized it because he, too, had been reversed. But he could never find the place again.

Getting your body flipped left to right would probably be fatal. Almost all of our body chemistry is chiral, so you would not be able to extract any nutrition from most foods, and you would starve to death or die of malnutrition.

If there is an extra dimension in which a wormhole exists, why not two extra dimensions? If there are two or more extra dimensions, you now have the issue described above, and you will need to make sure you get the two ends of your wormhole attached with the right geometry, or things that move through the wormhole might not come out quite as expected.

Of course, a black-hat could surely come up with evil things that could be done with that kind of wormhole.

When considering wormhole geometry, another potential problem is the curvature of space in the wormhole. According to Einstein's Theory of General Relativity, curved space causes uneven acceleration. Too much curvature can lead to disastrous gravitational tidal effects that can tear things apart. Small wormholes would be most likely to have this problem. Larger wormholes, like South Pass through the Rockies, would allow that curvature to be spread out enough to be hardly noticeable.

In what reference frame is traversal of the wormhole instantaneous?

This is the issue which to me is the killer.

Einstein's Theory of Special Relativity is quite well supported by experimental evidence. According to that theory, there is no such thing as universal simultaneity, so we have to ask what instantaneous travel means.

You may have heard that, according to Special Relativity, if observer A with clock A in spaceship A is moving near the speed of light relative to observer B, clock A will run more slowly than observer B's clock B, according to observer B, due to time dilation. But at the same time, according to observer A, observer B with clock B is moving near the speed of light relative to A, so observer A sees clock B as moving more slowly. This effect is the core of the twin paradox, where one twin gets on a spaceship from Earth, flies away at near light speed, and returns, while the other stays on Earth.

The twin paradox is resolved by noting that there is an asymmetry between the twins: one stays at rest on Earth, whereas the other accelerates three times during the trip (takeoff, turnaround, and landing). This difference is the key to understanding the paradox and determining that the twin on the spaceship ages more slowly than the one left on earth.

In 1971 a couple of scientists ran an experiment where they took some atomic clocks with them on commercial flights around the world and confirmed that they really did slow down as compared to the stationary atomic clocks left behind, just as predicted by Special Relativity (and by General Relativity, which predicted time dilation due to gravitational differences).

For instantaneous travel between wormholes, it seems like we can set up a symmetric situation so that we can't resolve our paradox the same way as for the twin paradox. Consider the situation where we have a wormhole between two spaceships (or planets, if you prefer) A and B that are moving at near the speed of light relative to each other. As noted above, the observer in each location observes the clock moving more slowly at the other location. If person C with clock C steps from spaceship A to B through the wormhole, spends a bit of time on spaceship B, then comes back to spaceship A, observer A will calculate that clock C will be behind clock A, having moved more slowly than clock A while it was on spaceship B. If person D with clock D steps from spaceship B to A through the wormhole, spends a bit of time on spaceship A, then goes back to spaceship B, observer A will calculate that clock D will be ahead of clock B, having moved more quickly than clock B while it was on spaceship A. But in this symmetric situation, observer B will calculate that clock C will be ahead of clock A, and clock D will be behind clock B, the opposite of what observer A calculates. So which is it?

The problem here is that statement that travel between wormholes is instantaneous. According to Special Relativity, two events that occur at the same time but different locations in one reference frame will occur at different times in a reference frame that is moving with respect to the first. For our example, this means that if observer A sees person C moving instantaneously through the wormhole from A to B, observer B does not see person C moving instantaneously through the wormhole except for when A and B are right next to each other. And since A and B are moving with respect to each other, they will not be right next to each other for at least one leg of the wormhole round trip. When A and B are not right next to each other, what appears as simultaneous in one reference frame is not simultaneous in the other reference frame.

The only way I know of that is consistent with Special Relativity that would allow wormhole travel to be instantaneous according to both ends of the wormhole would be to constrain wormholes to be stationary relative to each other. But this would be a pretty strong constraint for stories, since essentially everything in the universe is moving relative to each other, and even the rotation of a planet is enough velocity variation to cause measurable time issues across the kind of distances wormholes sometimes connect.

But wait, it gets crazier. By the laws of Special Relativity, if you have any mechanism that lets you move between two points faster than the speed of light, in any arbitrary frame of reference, you can use that mechanism to travel backwards in time. The Tachyonic antitelephone is an example of how being able to send a message faster than light allows sending a message backwards in time, and this same principle applies to sending an object rather than a message.

One way to explain this is based on the assertion of Special Relativity that two events that are not at the same location in space that occur simultaneously in a frame of reference A will not be simultaneous in a frame of reference B that is moving with respect to A. In frame B, one of those two events will happen before the other. Let's assume that we have a wormhole with a pair of distant portals that are stationary in frame A, and another wormhole with portals stationary in frame B, moving with respect to frame A in the direction from one of the A portals to the other. We arrange the portals such that wormhole portal B2 is immediately adjacent to wormhole portal A2 at the starting time of our experiment according to observer A located at A1, and we arrange that B1 and B2 are adjacent to A1 and A2, respectively, at the same time in frame B. At the starting time in A, we step from portal A1 to A2. Since we arranged for B2 to be adjacent to A2 at this time, we can immediately move over to B2 and step through to B1, which we assume is instantaneous in frame B. Because we have arranged that B1 is adjacent to A1 at the same moment as B2 is adjacent to A2 in frame B, when we exit B1 we can then hop back over to A1 and complete our circuit in space. Since our trip through the wormhole B is instantaneous in frame B, it will not be instantaneous in frame A. For the traveler, all four legs of the trip are nearly instantaneous, but for an observer who remains in A only three legs are, with the leg through wormhole B not being instantaneous. Depending on which direction travelers takes around this loop, they will return to A1 either well after or well before the time they left.

The amount of time is proportional to the distance traveled through the wormholes and is related to the velocity of one frame with respect to the other. If frame B is traveling near the speed of light relative to A, the amount of time will be close to the light-distance between the two ends of the portal, so even if you are "just" traveling to Proxima Centauri B near Alpha Centauri, the closest extrasolar star group to Earth at four light years away, you could travel up to four years into the future or the past. The effect is less pronounced, but still present, at lower speeds.

Note that Special Relativity itself doesn't preclude faster-than-light messages or travel, it just says that being able to do so allows sending a message or traveling backwards in time, as demonstrated above. Our current theories do not say this is not possible, but most people believe in causality and thus find time travel problematic.

If you want to get a better intuitive feel for some of the weird things that happen when you start moving at near the speed of light, check out the free video game A Slower Speed of Light from MIT.

Potential Answers

Given that typical science fiction wormholes are based on new science beyond our current theories, we have a lot of leeway in deciding how that science works so as to create the conditions that best advance our story. We could say that managing wormholes requires an amount of money and energy that are only available to large organizations, or we could say that, once the science is known, wormholes are easy and cheap and anybody can make them, and see what kind of havoc is wreaked. We could say that small wormholes are easy to make, or that larger wormholes are easier. We could choose the geometry of the wormhole and portals to be troublesome or trivial. We could say that wormhole portals require equipment to maintain, or that we can cast them anywhere with ease.

All of the above choices are pretty easy in the sense that they are about the fictional new wormhole science and don't conflict with our existing science. Things get a little harder when we try to decide how conservation of energy and momentum work with wormholes, but even there we should be able to postulate something that allows us to remain consistent with known science, such as the wormhole absorbing or supplying the difference, or perhaps even requiring an exchange of equal mass from either end of the wormhole.

Propagation of gravity through a wormhole seems to me a little more difficult to deal with. As mentioned above, you might be able to claim that wormhole technology allows controlling the curvature of space. But another view of mass and space is that mass is the curvature of space, in which case making space curve is equivalent to creating mass, and at that point we get into all the questions of conservation of mass and energy and where it comes from when curving space for a wormhole.

The one that I really can't figure out how to make consistent is, as mentioned above, the question of time. The main reason wormholes are typically introduced is to allow faster-than-light travel, which, as described above, is what leads directly to the potential of time travel, according to Special Relativity. For all of the other questions, it seems like it may be possible to define some new science that answers those questions in a way that does not require us to discard any of our current well-established scientific theories, but for faster-than-light travel, I don't see any way to do this.

I can't even just assume that Special Relativity doesn't apply in that universe. There is a deep connection between having the same laws of physics everywhere, electromagnetism, and having a maximum velocity for any matter or information. Special Relativity builds on the work of Newton and Maxwell. and discarding it would require some other significant changes to the way the universe works.

A science fiction author might choose to focus on how wormholes allow time travel, as Robert L. Forward does in some of his stories. For the other stories, the ones that don't mention time travel, I just have to suspend my understanding of Special Relativity and enjoy the story as told.

Golang Web Server Auth

An example of authentication and authorization in a simple web server written in go.

Contents

• Background
• Before Auth
• Adding Authentication
• Adding Authorization
• Summary

Background

As described in my previous blog post, I recently rewrote my image viewer desktop app as a web app, for which I wrote the web server in go.

Since I was adding a new potential attack vector, I wanted to add security; but since this is only available on my internal network, and it's not critically valuable data, I did not need enterprise-grade security. In this post I describe how I implemented a relatively simple authentication and authorization mechanism, in particular highlighting the features of go I used that made that easy to do. For a simple app such as this one, the third of the three As of security, auditing, can be done with simple logging if desired.

The code I present here is taken from the github repo for my mimsrv project, with links to specific commits and versions of various files. You can visit that project if you'd like to see more of the code than I present in this post.

Before Auth

Go has good support for writing simple web servers. The net.http package allows setting up a web server that routes requests based on path to specific functions. In the first commit for mimsrv, before there was any code for authentication or authorization, the http processing code looked like this:

In mimsrv.go:

func main() {
  ...
  mux := http.NewServeMux()
  ...
  mux.Handle("/api/", api.NewHandler(...))
  ...
  log.Fatal(http.ListenAndServe(":8080", mux))
}

In api/api.go:

func NewHandler(c *Config) http.Handler {
  h := handler{config: c}
  mux := http.NewServeMux()
  mux.HandleFunc(h.apiPrefix("list"), h.list)
  mux.HandleFunc(h.apiPrefix("image"), h.image)
  mux.HandleFunc(h.apiPrefix("text"), h.text)
  return mux
}

func (h *handler) list(w http.ResponseWriter, r *http.Request) {
  ...
}

The above two functions set up the routing and start the web server. The code in mimsrv.go creates a top-level router (mux) that routes any request with a path starting with "/api/" to the api handler that is created by the NewHandler function in api.go. The top-level router also defines routes for other top-level paths, such as "/ui/" for delivering the UI files.

The api code in turn sets up the second-level routing for all of the paths within /api (the h.apiPrefix function adds "/api/" to its argument). So when I make a request with the path /api/list, the main mux passes the request to the api mux, which then calls the h.list function.

Adding Authentication

To implement authentication in mimsrv, I added a new "auth" package with three files, and modified mimsrv.go to use that new auth package. The most interesting part of this change is that it implements the enforcement of the constraint that all requests to any path starting with "/api/" must be authenticated, yet I did not have to make any changes to any of the api code that services those requests.

When I originally wrote my request routing code, it could have been simpler if I had defined everything in one mux. I didn't do that because I think the approach I took provides better modularity, but in addition, that structure made it easy for me to require authentication for all of the api calls.

The authentication code itself is not trivial, but wiring that code into the request routing to enforce authentication for whole chunks of the request path space was. I wrote a wrapper function and inserted it in the middle of the request-handling flow for requests where I wanted to require authentication.

To wire in the authentication requirement for all requests starting with "/api/", I changed mimsrv.go to replace this line:

mux.Handle("/api/", api.NewHandler(...))

with these lines:

apiHandler := api.NewHandler(...))
mux.Handle("/api/", authHandler.RequireAuth(apiHandler))

Here is the RequireAuth method from the newly added auth.go:

func (h *Handler) RequireAuth(httpHandler http.Handler) http.Handler {
  return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request){
    token := cookieValue(r, tokenCookieName)
    idstr := clientIdString(r)
    if isValidToken(token, idstr) {
      httpHandler.ServeHTTP(w, r)
    } else {
      // No token, or token is not valid
      http.Error(w, "Invalid token", http.StatusUnauthorized)
    }
  })
}

The RequireAuth function looks at a cookie to see if the user is currently logged in (which means the user has been authenticated). If so, RequireAuth calls the handler it was passed, which in this case is the one created by api.NewHandler. If not, then RequireAuth calls http.Error, which prevents the request from being fulfilled and instead returns an authorization error to the web caller. When the mimsrv client gets this error it displays a login dialog.

The other code I added handles things like login, logout, and cookie renewal and expiration, but all of that code other than RequireAuth is specific to my implementation of authentication. You could instead, for example, use OAuth to authenticate, in which case you would have a completely different mechanism for authenticating a user, but you could still use a function similar to RequireAuth and wire it in the same way.

Adding Authorization

Wrapping selected request paths as described above makes it so that authentication provides authorization for those requests. This coarse-grained authorization is a good start, but for mimsrv I wanted to be able to use fine-grained authorization as well. As this is a simple program with a very small number of users, I don't need anything sophisticated such as role-based authorization. I chose to implement a model in which I only define permissions for global actions, then assign those permissions directly to users.

For this simple permissions model, I needed to be able to define permissions, assign them to users, and check them at run-time before performing an action that requires authorization. My permissions are simple strings, stored in a column in the CSV file that defines my users. To give a permission to a user, I manually edit that CSV file, and to check for authorization before taking an action, the code looks for that permission string in the set of permissions for the current user.

The one piece that is not obvious is how to pass the user's permissions to the code that needs to check them. The reason this is not obvious is because the http routing package defines the function signature for the functions that process an http request, and that function signature includes only the request and a writer for the response. You can't simply add another argument in which you pass your user information, so you have to dig a little deeper to figure out how to pass along that information.

The solution relies on the fact that there is a Context attached to the Request that is passed to the handler function. By adding the user info to the Context, you can then extract that information further along in the processing when you need to check the permission.

The RequireAuth function validates that the user making the request is authenticated, so it already has information about who the user is, and this is the point at which we want to add the user info to the Context. We do this in our RequireAuth function by replacing this line:

      httpHandler.ServeHTTP(w, r)

with these lines:

      user := userFromToken(token)
      mimRequest := requestWithContextUser(r, user)
      httpHandler.ServeHTTP(w, mimRequest)

func requestWithContextUser(r *http.Request, user *users.User) *http.Request {
  mimContext := context.WithValue(r.Context(), ctxUserKey, user)
  return r.WithContext(mimContext)
}

When the code needs to know whether the current user is authorized for an action, it can call the new CurrentUser function, which retrieves the user info from the Context attached to the Request, from which the code can query the user's permissions:

func CurrentUser(r *http.Request) *users.User {
  v := r.Context().Value(ctxUserKey)
  if v == nil {
    return nil
  }
  return v.(*users.User)
}

Summary

While implementing authentication and authorization in a web server takes more than just a few lines of code, at least the part about how it gets tied in to the http processing in go is only a few lines. Although that part is only a few lines of code, it took me a while to dig around and find exactly how to do that. I hope that this article can save some other people a bit of time when doing their own research on how to add auth to a go web server.