Short question: should the resulting quaternion between rotation of two vectors (0, 0, 1) to (0, 0, -1) being a undefined values (nan)? Or this is a bug.
local v1 = vmath.vector3(0, 0, 1)
local v2 = vmath.vector3(0, 0, -1)
local rot = vmath.quat_from_to(v1, v2)
print(rot) -- vmath.quat(-nan, -nan, -nan, 0)
The vectors are directly opposing, which isn’t allowed.
Perhaps we should fix this and instead of nan return an identity quat? This is what Three.js does (https://github.com/mrdoob/three.js/blob/dev/src/math/Quaternion.js#L352). I’m not sure what other engines do?
If it’s going to change, I would just return nil or false. It can’t possibly give a specific value because it doesn’t know what plane to rotate around, so you’ll always need to check for it. nil or false would at least be easier to handle than a quat with NaN values, whereas an identity quat would mislead you into thinking that no rotation was necessary. 
You can check for nan by comparing it to itself. It will not be equal to itself.
if rot.x ~= rot.x then ... (is NaN)
yep, this comparing was in my code, but I don’t really like post-checking. Besides, sometimes the results were not nan but inf or -inf.
now using that code:
if vmath.dot(v1, v2) ~= -1 then
local q = vmath.quat_from_to(v1, v2)
...