Yes, today I dropped the first "NO" on a senior faculty member. Since the time I have started working as a TT faculty I have been pulled at various directions by various faculty members. Some of these directions I want to be pulled at because it is essential for me to set up here and get the ground going. But other kinds of requests or possible collaborations are just too much stress on my time.
I just politely told said a No to a possible collaboration. Although it would have been a fun thing to do it was mostly teaching related and I do not see it enhancing my career forward as of now.
Now I see what all other new TT's have been complaining about and advising how to say No's to multiple people. This advice is right on the money. Otherwise you might get pulled in too many directions to handle.
So my first advice as a new TT faculty: Learn to say NO (but in a nice way so that you don't offend people who are ultimately going to determine your tenure :D)