I think, that it is still obtaining more items like that than you are supposed to have, which is the main reason island farming is illegal, thus making this fall into that category.
I know the rules regarding "borrowing" dirt are much more grey. Some Mods are OK with it, some aren't. Alot of it has to do with the situation too. If you're a known upstanding member of the community who needs to make a tree farm for a project, and "borrowed" some dirt from soneone at no charge whatsoever, and later was shown to return it, it would look alot different from someone who was previously banned 3x for island farming who is now claiming to "borrow" dirt for a farm. I know that's not a cut and dry explaination, but legal stuff isn't cut and dry, it's why we have Mods not trade filters. Technically the term trade implies a 2 way transaction, IE, you're giving something for something else in return. But that something could just as well be nothing, IE, you trade 32 dirt for an air block, or a tour of an island. Borrowing implies that the item be returned, however, one can also "borrow" an item indefinitely, which makes it hard to quantify and enforce. A simpler way to phrase it is, any dirt that is on your island or in your inventory anywhere should have come from 1) Dirt that was originally on the island 2) Grass blocks that were legally obtained (Trade, Donation, Voting) to be on the safe side. Dirt obtained from anywhere else should be cleared by a Mod first.