Programming with Interfaces

Today I was told that:

In general, interfaces should be used sparingly as they add unneeded complexity when applied to a simple set of objects.

I don’t get it, I thought interfaces were a Good Thing. I must be missing something….