The Impact of Cloud Computing on Data Engineering: Leveraging AWS, Google Cloud, and Azure
DOI:
https://doi.org/10.63282/3050-9246.IJETCSIT-V4I3P118Keywords:
Cloud Computing, Data Engineering, AWS, Google Cloud, Microsoft Azure, Big Data, Machine Learning, Data Warehousing, Data Security, Scalability, Data Processing, Cloud InfrastructureAbstract
Cloud technology drives the data engineering transformation because businesses achieve operational changes through different levels of efficiency. An evaluation examines the data engineering process changes by investigating major cloud solutions which include AWS and GCP and Microsoft Azure. The complete extraction of database content values occurs through platform applications that follow collection phases followed by storage steps and finish with analytical and security functions. Data engineers now use modernized standard on-site systems which change into flexible processing systems powered by cloud technology for big data management. Engineers build dependable and cost-efficient data pipeline frameworks by uniting Cloud serverless constructs with managed solution features and ML capabilities from Cloud services. The analysis provides insights about operational features present in AWS GCP and Azure services and it presents advantages and disadvantages of these services. The pace of data engineering transformation has accelerated because artificial intelligence brings together real-time processing of big data analytics together with complex data warehousing and several advanced technologies. New business operations find benefits from this innovation because it transforms data into quicker decision-making tools for leadership functions that process large data volumes. The analysis system built in cloud data engineering produces an effective framework to forecast data engineering system operational states in present times and upcoming periods.
Downloads
References
[1] M. Armbrust et al., "A view of cloud computing," Commun. ACM, vol. 53, no. 4, pp. 50–58, Apr. 2010.
[2] J. Casalderrey-Solana et al., "A performance analysis of serverless computing platforms," in Proc. IEEE Int. Conf. Cloud Eng. (IC2E), 2019, pp. 30–35.
[3] S. Sakr and A. Y. Zomaya, Encyclopedia of Big Data Technologies. Cham, Switzerland: Springer, 2019.
[4] V. G. K. Reddy, "Comparative study of cloud computing services: AWS, Microsoft Azure and Google Cloud," Int. J. Innov. Technol. Explor. Eng., vol. 8, no. 6, pp. 2039–2044, 2019.
[5] L. B. H. M. van der Vlist, "Modern data engineering: A review on the transition from on-premise to cloud-native data pipelines," J. Big Data, vol. 7, no. 1, p. 45, June 2020.
[6] J. Dean and S. Ghemawat, "MapReduce: Simplified data processing on large clusters," Commun. ACM, vol. 51, no. 1, pp. 107–113, Jan. 2008.
[7] R. Sridhar, "A novel fusion algorithm for load balancing to maximize response time in cloud computing," Int. J. Multidiscip. Res., vol. 1, no. 3, Art. no. 52025, Nov.–Dec. 2019. [Online]. Available: https://www.ijfmr.com/research-paper.php?id=52025
[8] R. Sridhar, "Security challenges and solutions in virtualized cloud infrastructures," Int. J. Multidiscip. Res., vol. 1, no. 3, Art. no. 52026, Nov.–Dec. 2019. [Online]. Available: https://www.ijfmr.com/research-paper.php?id=52026
[9] R. Sridhar, "High availability strategies in cloud infrastructure management," Int. J. Multidiscip. Res., vol. 3, no. 2, Art. no. 52042, Mar.–Apr. 2021. [Online]. Available: https://www.ijfmr.com/research-paper.php?id=52042
[10] R. Sridhar, "Optimizing cloud performance: A comparative survey of load balancing algorithms," Int. J. Multidiscip. Res., vol. 3, no. 5, Art. no. 52046, Sept.–Oct. 2021. [Online]. Available: https://www.ijfmr.com/research-paper.php?id=52046
[11] R. Sridhar and I. J. Kanani, "Automated detection and prevention of deepfake content in digital news reporting," World J. Adv. Res. Rev., vol. 14, no. 3, pp. 890–894, June 2022.
[12] R. Sridhar, "Enhancing website security through prevention: SQL and data inference injections," World J. Adv. Res. Rev., vol. 14, no. 3, pp. 841–847, June 2022.
[13] R. N. K. Dhenia and R. Sridhar, "The impact of data bias on decision making," World J. Adv. Res. Rev., vol. 14, no. 3, pp. 848–852, June 2022.
[14] R. N. K. Dhenia, "Text mining and social media analysis for mental health insights," World J. Adv. Res. Rev., vol. 15, no. 3, pp. 640–645, Sept. 2022.
[15] R. Sridhar, "The evolving landscape of the internet of things: A review of modern technologies, applications, and core challenges," World J. Adv. Res. Rev., vol. 15, no. 3, pp. 646–651, Sept. 2022.
[16] R. Sridhar, "Preserving architectural integrity: Addressing the erosion of software design," Int. J. Sci. Res., vol. 9, no. 12, pp. 1939–1944, Dec. 2020. doi: 10.21275/MS2012134218.
[17] R. Sridhar and R. N. K. Dhenia, "An analytical study of NoSQL database systems for big data applications," Int. J. Sci. Res., vol. 9, no. 8, pp. 1616–1619, Aug. 2020. doi: 10.21275/MS2008134522.
[18] R. Sridhar, "Leveraging open-source reuse: Implications for software maintenance," Int. J. Sci. Res., vol. 9, no. 2, pp. 1969–1973, Feb. 2020. doi: 10.21275/MS2002134347.
[19] I. J. Kanani, "Security misconfigurations in cloud-native web applications," Int. J. Sci. Res., vol. 9, no. 12, pp. 1935–1938, Dec. 2020. doi: 10.21275/MS2012131513.
[20] I. J. Kanani and R. Sridhar, "Cloud-native security: Securing serverless architectures," Int. J. Sci. Res., vol. 9, no. 8, pp. 1612–1615, Aug. 2020. doi: 10.21275/MS2008134043.
