The servant was looking for fun with his owner when her husband was out