Nissan Altima 2005 possible tran problem?

This has only happened twice.. The first time I was reversing out of my driveway and my car stalled, so I did what I guess you would normally do put it in park, restart it etc. But when I put it into drive it would not go, it was like I was in neutral (this was probably a month ago). The most recent time was yesterday, I was driving everything was fine. Pulled into a 711, got out, came back went to start my car, it hesitated like it didn't want to start but it did. Reversed fine, put it in drive, and it wouldn't go it was like that being in neutral again. If I pushed hard enough on the gas it went but it was like something was stopping it. No lights were on, no signs of anything. But what I did notice was when I put it into reverse a huge clunk happened, going back into drive still no luck. I also noticed that when I put it over into I guess the manual way it wasn't on 1, it was in 5. I made it back to where I was going parked the car, tran fluid was good. This morning had no problems drove over 100 miles. I'm a little concerned though, I'm not really sure where to start it's a 2005 Altima and it's not like it's happening everytime. help?