Normally that shouldn't come loose.
Would go for a complete chain kit with proper installation.
There are a number issues that come to mind, not sure whether any of them would loosen the nut though...
... unevenly lengthened chain, poor wheel alignment, incorrect chain tension, crooked or not...