Cloud computing is now considered to be the most cost-effective platform for offering business and consumer IT services over the Internet. However, it is prone to new vulnerabilities. Specifically, a newly discovered type of attack, called an economic-denial-of-sustainability attack known as EDoS, exploits the pay-per-use model to scale up the resource usage over time to the degree that the cloud user has to pay for the unexpected usage charge. To prevent EDoS attacks, we propose an effective solution in the SDN-based cloud computing environment. We first introduce a machine-learning-based approach adopting a framework called MAD-GAN which applies an unsupervised multivariate anomaly detection technique based on Generative Adversarial Networks (GANs), using the Long-Short-Term-Memory Recurrent Neural Networks (LSTM-RNN) to detect EDoS attacks. Its main idea is to produce an anomaly score at each time step by learning a multivariate attribute. We then generate a dynamic threshold score to compare with the anomaly score produced to classify the network traffic as EDoS traffic or normal traffic. By realistic tests, our proposed scheme is demonstrated to outperform existing methods for EDoS attack detection. The detailed experiments conducted with different levels of EDoS attacks show that the proposed scheme is an efficient, innovative approach to defend EDoS attacks in the SDN-based cloud.