Cousin violating boundaries as a child.

When I was a boy of about 6 years old, my cousin and I became good friends. He is 1.5 years older than me. When I was 6, we spent a holiday at our grandparents' place, and one morning when we were in bed (we shared a bedroom), he suggested we play game in which he would put his penis between my buttocks. I first said no, arguing that our grandparents may walk in. He argued they were still asleep. I let him do it, but felt uncomfortable, and told him to stop, which he did. We had a good time together the rest of the holiday, and remained friends for some years after this, and there was no more of such attempts from his side. We drifted apart, but would still see each other at family events. All the time, I remembered what had happened but wasn't affected by it. Later, in my early 20s, I saw a therapist because of anxiety, and we talked a great deal about things from my childhood that caused me more distress, particularly bullying which took place at school a bit later than the incident at my grandparents. I didn't bring up the incident, because it didn't matter that much to me. I don't think it was about denial, because we talked about a lot of things that were distressful and a I had the feeling of "cleaning out my closet". I am now 49 years old. Six months ago my cousin re-contacted because he and his family would be on holiday close to where we have our summer cottage. It was nice to see him, because I recalled our friendship, and our daughters (both are 11 years old) got on really well. We invited him back, and a month ago he and his daughter spent a weekend at our place. But a week ago, I started thinking a lot more about when he crossed my boundaries, and how to deal with it. I also worry that he may sexually abuse his own children or my daughter if she spends time alone with him, but don't know how to deal with or what to do.