When I found myself in a similar situation I decided to douse it. I feel so strongly about being faithful (both physically and emotionally) to my sweet DH that I decided I was going to get rid of the emotions to make sure nothing of the physical would EVER happen. ALL affairs start with emotions.
What did I do? Well, DH and I had a very open discussion about how we both need to be careful. I told him about the struggle with my emotions with the doc and reiterated my determination to be faithful in all ways. I decided that I needed to remove the situation from my life and promised DH that I wouldn't schedule with that doctor anymore.
But I was still struggling. So I did the ultimate douser - I called the doc. I told him that I wasn't going to be scheduling with him anymore - not that he was a bad doc, but because I was too attracted to him. (Pretty gutsy huh? ;)
I did this so that he would know he needed to be careful around me. (He was starting to become more casual with me and had told me that he and his wife had just separated.) I also didn't want to change practices, and may have to see him again. I needed to make sure that we could have a good working relationship.
You know what? After that phone call, the internal struggle stopped. I had taken steps to make sure nothing COULD ever happen. It was over and I felt so peaceful. The fight to protect my emotions and my family was so worth it. I NEVER want to put my family through what affairs do and I will fight forever to be faithful!