The basis of your internal conflict concerning this very personal matter (which you are to be commended in that you are willing to reveal it brothers and sisters, in Christ, who care) seems to me to be;
1.) you desire to bear fruit for God in your life (not fruit that is subject to death and corruption),
2.) you desire to serve God (not serve the "old", and find yourself and others in complicated circumstances that tend to produce labor and hardship).
These difficult circumstances may very well be the Lord's drawing you closer to Himself, as He does with all of us.
This is why I am inclined, when discussing such matters as marriage, and divorce between human persons, to refer again to Romans 7 as a sure guide to help us decide and act.
First off, the death of Christ, as we are participants in it as explained by Paul in Romans 6, means that we are "dead to the law by the body of Christ" (verse 4). Paul uses the human institution of marriage to explain this.
We, as humans, are tied to the old humanity by "law". Through the death of Christ, now, we are "dead to the law".......so that we may "joined" to another.
The propriety of joining to another in the outward physical realm
should be preceded by....the being "joined to another" in a spiritual sense.
In my opinion, the Lord is using outward circumstances to focus on inward truths. That is what is truly important.
Further, we are to see ourselves as being "delivered from the law" that held us as humans, whereby we "should serve in newness of spirit, and not in oldness of letter" (verse 6).
In the final analysis, "the oldness of the letter" which may dictate certain specific actions of the flesh (...i.e...Moses "letter" concerning divorce/adultery) is not to be our realm of service any longer.
May God grant us the wisdom that we need to serve Him, knowing that only He can bear fruit through us.
For I am persuaded, that neither death, nor life, nor angels, nor principalities, nor powers, nor things present, nor things to come, Nor height, nor depth, nor any other creature, shall be able to separate us from the love of God, which is in Christ Jesus our Lord. Romans 8:38,39