Exactly. It only works if the light source (or the camera in case of light tracing) are intersectable.
The normal way in ray tracing is this: You shoot a ray into the scene, then you check which object the ray intersects. From that intersection point, you check if the light source is directly visible - i.e. no other object lies between them - and if yes, calculate the brightness of that ray according to what the probability is that the ray would bounce in that direction, depending on how matte or glossy the surface is.
With a perfeclty specular material, the probability is always 0 except in one direction. Then the above is not possible, because you can't know easily for arbitrary geometry by which path a ray would connect.
So therefore, a ray path ending with Specular can only happen by random intersection, which is only efficient if the light source appears large from that object, or is something continuous like an HDRI background.
SDS paths are so special because they start and end with specular, so coming form either camera (path tracing) or light (light tracing), it is equally impossible.
BiDir starts tracing from both directions and tries to connect them. For this algorithm,the problem the becomes that both rays would have to randomly end up on the exact same point on the diffuse surface, which is equally unlikely.
Metropolis sampling does not change that.