It's interesting how we are not as imaginative as we might think; as you quoted from Solomon, "There is no new thing under the sun."
This issue that you mentioned above reminds me of the account of Lot when he first moved his family to the plains of Jordan. While living in that area was not inherently sinful, he made the choice to go there without consulting God, and it eventually lead him to Sodom - and ultimately to the loss and moral and spiritual perversion of his family.
Men can and do often overreact to things. I'm not saying our predecessors were wrong for accepting all the music they did, but neither am I saying they were right. There were undoubtedly many reactionaries that shied away from "new music" in the 18 and 1900's because they were suspicious of anything that was new. Being cautious is nothing to scoff at - as I'm sure you realize - but neither is examining and accepting that which is truly of God.
I would say the same of the ones you've referred to as the "old guard." I have no doubt that there were a good amount of reactionaries involved there as well and again, I'm not saying that they were wrong to accept all the music they did, but neither am I saying they were right.
The same applies to today. There are many reactionaries who are simply "reacting" to CCM because it is "new." They don't have any other reason for opposing it other than that it is "new." That is a faulty base premise for rejecting something - which you obviously understand.
My point is that music's "newness" or "oldness" is not a relevant reason for using or not using it - in the church, or in our personal lives. The age of the music does come in to play when understanding the historical influences of the period the writer(s) lived in, but it is not a sound method for final-decision-making.
Simply because music has been accepted "for years," or is "traditionally appropriate," doesn't validate its use. Yes, our predecessors may have accepted a song one hundred, two hundred, twenty, or two years ago... and if they used their own judgement to accept it, they may have been wrong. The true test for music is to be God's Word, not man's:
I Thessalonians 5:21: "Prove all things; hold fast that which is good."
Romans 12:1-2: "I beseech you therefore, brethren, by the mercies of God, that ye present your bodies a living sacrifice, holy, acceptable unto God, which is your reasonable service. And be not conformed to this world: but be ye transformed by the renewing of your mind, that ye may prove what is that good, and acceptable, and perfect, will of God."
Well, fortunately for the vast portion of Christians, you don't have to be a "musician" to understand what Godly music is - you don't even have to know anything about music theory. If we really want to know the truth about what God wants, He will guide us to that truth:
John 16: 13: "Howbeit when he, the Spirit of truth, is come, he will guide you into all truth: for he shall not speak of himself; but whatsoever he shall hear, that shall he speak: and he will shew you things to come."
II Peter 1:2-4: "Grace and peace be multiplied unto you through the knowledge of God, and of Jesus our Lord,According as his divine power hath given unto us all things that pertain unto life and godliness, through the knowledge of him that hath called us to glory and virtue:Whereby are given unto us exceeding great and precious promises: that by these ye might be partakers of the divine nature, having escaped the corruption that is in the world through lust."
I think the best way I know of to explain what God-honouring music "looks" like is to take you back to the basic definition of music - the one I've set forth: "An expression of the inner man."
As you know, no man can please God in and of himself - it's against his nature.
Jeremiah 17:9-10: "The heart is deceitful above all things, and desperately wicked: who can know it?I the Lord search the heart, I try the reins, even to give every man according to his ways, and according to the fruit of his doings."
Genesis 8:21: "And the Lord smelled a sweet savour; and the Lord said in his heart, I will not again curse the ground any more for man's sake; for the imagination of man's heart is evil from his youth; neither will I again smite any more every thing living, as I have done."
Romans 3:9-11: "What then? are we better than they? No, in no wise: for we have before proved both Jews and Gentiles, that they are all under sin;As it is written, There is none righteous, no, not one:There is none that understandeth, there is none that seeketh after God.
When a person is "saved" the Holy Spirit comes to dwell in him and that person receives a "new nature" - his inner man is changed.
Colossians 3:8-10 (really the rest of the chapter, but I'll not post the whole thing for sake of space): "But now ye also put off all these; anger, wrath, malice, blasphemy, filthy communication out of your mouth.Lie not one to another, seeing that ye have put off the old man with his deeds;And have put on the new man, which is renewed in knowledge after the image of him that created him:
That new nature manifests itself in many ways:
Galatians 5:22-23: "But the fruit of the Spirit is love, joy, peace, longsuffering, gentleness, goodness, faith,Meekness, temperance: against such there is no law."
James 3:17-18: "But the wisdom that is from above is first pure, then peaceable, gentle, and easy to be intreated, full of mercy and good fruits, without partiality, and without hypocrisy.And the fruit of righteousness is sown in peace of them that make peace."
Romans 12:1-2 (and the rest of chapter 12): "I beseech you therefore, brethren, by the mercies of God, that ye present your bodies a living sacrifice, holy, acceptable unto God, which is your reasonable service.And be not conformed to this world: but be ye transformed by the renewing of your mind, that ye may prove what is that good, and acceptable, and perfect, will of God."
The inner man houses our beliefs - about God, life, ourselves, everything really. When a man's view/belief of God is skewed, the way he expresses his inner man will naturally reflect that problem - rather like having a crooked foundation for a building.
So, to answer your question, Godly music is produced by an inner man that knows God and seeks to follow and obey Him the way God instructs him to in His Word. How is that seen by a layman? By looking at the beliefs and life/lives of the one(s) whose inner man/men produced the music.
Matthew 7:15-23: "Beware of false prophets, which come to you in sheep's clothing, but inwardly they are ravening wolves.Ye shall know them by their fruits. Do men gather grapes of thorns, or figs of thistles?Even so every good tree bringeth forth good fruit; but a corrupt tree bringeth forth evil fruit.A good tree cannot bring forth evil fruit, neither can a corrupt tree bring forth good fruit.Every tree that bringeth not forth good fruit is hewn down, and cast into the fire.Wherefore by their fruits ye shall know them.Not every one that saith unto me, Lord, Lord, shall enter into the kingdom of heaven; but he that doeth the will of my Father which is in heaven.Many will say to me in that day, Lord, Lord, have we not prophesied in thy name? and in thy name have cast out devils? and in thy name done many wonderful works?And then will I profess unto them, I never knew you: depart from me, ye that work iniquity."