For this problem, I don't think you can do it without any unfolding, just minimize the amount you have to do. I say this because you have to visualize the unfolding process a little bit to know how the triangle's dimensions should more or less come out to be. Also, there isn't really a formula that one can use because the distances you use as your triangle legs will vary depending on the specific position of the points. However, for this specific problem, the formula would be sqrt[(w + h)^2 + l^2].
You are correct in saying that the longest diagonal line is sqrt(115); the answer to the question does not suggest anything to the contrary. The aforementioned value is the distance between points A and B when you go through the rectangular prism. You are trying to find the shortest distance going along the surface of the object
Choices D and E are the distances between the two points when one calculates the hypotenuse of one of the faces then goes along the edge.