I've heard it said that pastors cannot have friends inside the church they serve. I've even heard seminarians recount how professors discouraged them from developing friendships with their people once they're in the pastorate. They were told that such friendships weren't possible or that they were bad ideas.
That's not been my experience in any of the churches I've had the honor of serving, either as an elder or main preaching pastor. The Lord has kindly given me friendships in the church--in fact, my main friendships.
Off the back of the conversations about accountability, one of the issues that I think many pastors face is that they just don't know others well enough to have such accountibility. If you have a mature group of elders within your congregation you may find one such brother with whom you can share openly. I am delighted that Dean, our youth pastor is a model of such openness within our eldership and makes it easy to be open back.