B. Sanou, ICT Facts and Figures, tech. rep, 2013.

M. Mills, The Cloud Begins With Coal, 2013.

, Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, tech. rep, 2017.

A. Fehske, G. Fettweis, J. Malmodin, and G. Biczok, The global footprint of mobile communications: The ecological and economic perspective, IEEE Communications Magazine, vol.49, issue.8, pp.55-62, 2011.

L. Belkhir and A. Elmeligi, Assessing ICT global emissions footprint: Trends to 2040 & recommendations, Journal of Cleaner Production, vol.177, pp.448-463, 2018.

, Energy Harvesting, 2014.

&. Solargis-maps,

, Fixed Tilt Solar Panel, p.10

B. Coxworth, Starting in 2020, solar panels will be required on all new California houses, p.10, 2018.

, Cars with Solar Panels, p.10, 2018.

S. Kim, R. Vyas, J. Bito, K. Niotaki, A. Collado et al., Ambient RF Energy-Harvesting Technologies for Self-Sustainable Standalone Wireless Sensor Platforms, Proceedings of the IEEE, vol.102, issue.11, p.11, 2014.

, PN junction -Solar Cell Central

, How Solar Works

J. Gruetter, Solar Energy Harvesting: Low Power in a Compact Footprint, p.11

S. Cao and J. Li, A survey on ambient energy sources and harvesting methods for structural health monitoring applications, Advances in Mechanical Engineering, vol.9, issue.4, p.16, 2017.

, Samsung EH phone comparison, p.12

, EH phone comparison, p.12

X. Iphone and . Tesla,

M. Hamid-elsheikh, D. A. Shnawah, M. F. Sabri, S. B. Said, M. Hassan et al., A review on thermoelectric renewable energy: Principle parameters that affect their performance, Renewable and Sustainable Energy Reviews, vol.30, pp.337-355, 2014.

D. Enescu, Thermoelectric Energy Harvesting: Basic Principles and Applications, IntechOpen, pp.13-14, 2019.

I. Sil, S. Mukherjee, and K. Biswas, A review of energy harvesting technology and its potential applications, Environmental and Earth Sciences Research Journal, vol.4, issue.2, p.16, 2017.

J. Chen, Design and Analysis of a Thermoelectric Energy Harvesting System for Powering Sensing Nodes in Nuclear Power Plant, 2015.

X. Zhang and L. Zhao, Thermoelectric materials: Energy conversion between heat and electricity, Journal of Materiomics, vol.1, issue.2, pp.92-105, 2015.

X. Wenyu, Experimental and Numerical Analysis of Thermoelectric MagnetoHydroDynamic Driven Liquid Lithium Flow in Open Channels for Fusion Applications, 2015.

Y. K. Ramadass and A. P. Chandrakasan, A batteryless thermoelectric energy-harvesting interface circuit with 35mv startup voltage, IEEE International Solid-State Circuits Conference (ISSCC), pp.486-487, 2010.

, Vodafone unveils the future of festival season tech: Charge your phone while you sleep, 2013.

W. K. Seah, Z. A. Eu, and H. Tan, Wireless sensor networks powered by ambient energy harvesting (WSN-HEAP) -Survey and challenges, IEEE International Conference on Wireless Communication, Vehicular Technology, Information Theory and Aerospace & Electronic Systems Technology, p.16, 2009.

C. Williams and R. Yates, Analysis Of A Micro-electric Generator For Microsystems, IEEE International Solid-State Sensors and Actuators Conference -TRANSDUCERS, vol.1, pp.369-372, 1995.

E. Hau, The Wind Resource, Wind Turbines, pp.505-547, 2013.

M. Ku, W. Li, Y. Chen, K. J. Ray, and . Liu, Advances in Energy Harvesting Communications: Past, Present, and Future Challenges, vol.18, p.29, 2016.

X. Lu, P. Wang, D. Niyato, D. I. Kim, and Z. Han, Wireless Networks With RF Energy Harvesting: A Contemporary Survey, IEEE Communications Surveys & Tutorials, vol.17, issue.2, p.17, 2015.

M. Pinuela, P. D. Mitcheson, and S. Lucyszyn, Ambient RF Energy Harvesting in Urban and Semi-Urban Environments, IEEE Transactions on Microwave Theory and Techniques, vol.61, issue.7, p.17, 2013.

L. R. Varshney, Transporting information and energy simultaneously, IEEE International Symposium on Information Theory, pp.1612-1616, 2008.

T. D. Perera, D. N. Jayakody, S. K. Sharma, S. Chatzinotas, and J. Li, Simultaneous Wireless Information and Power Transfer (SWIPT): Recent Advances and Future Challenges, IEEE Communications Surveys & Tutorials, vol.20, issue.1, pp.264-302, 2018.

P. Mach and Z. Becvar, Mobile Edge Computing: A Survey on Architecture and Computation Offloading, IEEE Communications Surveys & Tutorials, vol.19, issue.3, p.29, 2017.

T. Taleb, K. Samdanis, B. Mada, H. Flinck, S. Dutta et al., On Multi-Access Edge Computing: A Survey of the Emerging 5g Network Edge Cloud Architecture and Orchestration, IEEE Communications Surveys & Tutorials, vol.19, issue.3, pp.1657-1681, 2017.

H. Hong, From Cloud Computing to Fog Computing: Unleash the Power of Edge and End Devices, IEEE International Conference on Cloud Computing Technology and Science (CloudCom), (Hong Kong), pp.331-334, 2017.

K. Dolui and S. K. Datta, Comparison of edge computing implementations: Fog computing, cloudlet and mobile edge computing, IEEE Global Internet of Things Summit (GIoTS), pp.1-6, 2017.

E. N. Gilbert, Capacity of a Burst-Noise Channel, Bell System Technical Journal, vol.39, issue.5, p.20, 1960.

A. Tolver, An Introduction to Markov Chain, p.20, 2016.

B. Givan and R. Parr, An Introduction to Markov Decision Processes, p.20

F. Spieksma and R. Nunez-queija, Markov Decision Processes, p.20, 2015.
URL : https://hal.archives-ouvertes.fr/inria-00074992

S. , Infinite Horizon Markov Decision Problems, 2016.

R. S. Sutton and A. G. Barto, Reinforcement learning: an introduction. Adaptive computation and machine learning, p.92, 1998.

E. Altman, Constrained Markov Decision Processes, p.27, 2004.
URL : https://hal.archives-ouvertes.fr/inria-00074109

S. Mahadevan, Average reward reinforcement learning: Foundations, algorithms, and empirical results, Machine Learning, vol.22, p.23, 1996.

D. P. Bertsekas, in Athena scientific optimization and computation series, vol.1, p.25, 2005.

P. Geibel, ;. Hutchison, T. Kanade, J. Kittler, J. M. Kleinberg et al., Reinforcement Learning for MDPs with Constraints, Machine Learning: ECML 2006, vol.4212, p.26, 2006.

S. Ulukus, A. Yener, E. Erkip, O. Simeone, M. Zorzi et al., Energy Harvesting Wireless Communications: A Review of Recent Advances, IEEE Journal on Selected Areas in Communications, vol.33, issue.3, p.29, 2015.

K. Kumar, J. Liu, Y. Lu, and B. Bhargava, A Survey of Computation Offloading for Mobile Systems, Mobile Networks and Applications, vol.18, issue.1, pp.129-140, 2013.

W. Wu, J. Wang, X. Wang, F. Shan, and J. Luo, Online Throughput Maximization for Energy Harvesting Communication Systems with Battery Overflow, IEEE Transactions on Mobile Computing, vol.16, issue.1, p.29, 2017.

B. T. Bacinoglu, E. Uysal-biyikoglu, and C. E. , Finite-Horizon Energy-Efficient Scheduling With Energy Harvesting Transmitters Over Fading Channels, IEEE Transactions on Wireless Communications, vol.16, issue.9, p.29, 2017.

A. O. Isikman, M. Yuksel, and D. Gunduz, A Low-Complexity Policy for Outage Probability Minimization With an Energy Harvesting Transmitter, IEEE Communications Letters, vol.21, issue.4, p.29, 2017.

L. Lei, Y. Kuang, X. S. Shen, K. Yang, J. Qiao et al., Optimal Reliability in Energy Harvesting Industrial Wireless Sensor Networks, IEEE Transactions on Wireless Communications, vol.15, issue.8, p.36, 2016.

I. Ahmed, K. T. Phan, and T. Le-ngoc, Optimal Stochastic Power Control for Energy Harvesting Systems With Delay Constraints, IEEE Journal on Selected Areas in Communications, vol.34, issue.12, p.30, 2016.

S. Barbarossa, S. Sardellitti, and P. Lorenzo, Communicating While Computing: Distributed mobile cloud computing over 5g heterogeneous networks, IEEE Signal Processing Magazine, vol.31, issue.6, p.30, 2014.

D. Huang, P. Wang, and D. Niyato, A Dynamic Offloading Algorithm for Mobile Computing, IEEE Transactions on Wireless Communications, vol.11, issue.6, p.30, 1991.

W. Zhang, Y. Wen, K. Guan, D. Kilper, H. Luo et al., Energy-Optimal Mobile Cloud Computing under Stochastic Wireless Channel, IEEE Transactions on Wireless Communications, vol.12, issue.9, p.30, 2013.

W. Labidi, M. Sarkiss, and M. Kamoun, Energy-optimal resource scheduling and computation offloading in small cell networks, IEEE International Conference on Telecommunications (ICT), p.36, 2015.
URL : https://hal.archives-ouvertes.fr/cea-01841678

O. Munoz, A. Pascual-iserte, and J. Vidal, Optimization of Radio and Computational Resources for Energy Efficiency in Latency-Constrained Application Offloading, IEEE Transactions on Vehicular Technology, vol.64, issue.10, p.30, 2015.

J. Liu, Y. Mao, J. Zhang, and K. B. Letaief, Delay-optimal computation task scheduling for mobile-edge computing systems, IEEE International Symposium on Information Theory (ISIT), p.30, 2016.

J. Zhang, X. Hu, Z. Ning, E. C. Ngai, L. Zhou et al., Energy-Latency Tradeoff for Energy-Aware Offloading in Mobile Edge Computing Networks, IEEE Internet of Things Journal, vol.5, issue.4, p.30, 2018.

Y. Mao, J. Zhang, and K. B. Letaief, Dynamic Computation Offloading for Mobile-Edge Computing With Energy Harvesting Devices, IEEE Journal on Selected Areas in Communications, vol.34, issue.12, p.36, 2016.

J. Xu, L. Chen, and S. Ren, Online Learning for Offloading and Autoscaling in Energy Harvesting Mobile Edge Computing, IEEE Transactions on Cognitive Communications and Networking, vol.3, issue.3, p.30, 2017.

M. Min, L. Xiao, Y. Chen, P. Cheng, D. Wu et al., Learning-Based Computation Offloading for IoT Devices With Energy Harvesting, IEEE Transactions on Vehicular Technology, vol.68, issue.2, p.30, 2019.

S. Kaul and R. Yates, Real-time status: how often should one update?, IEEE International Conference on Computer Communications (INFOCOM), p.31, 2012.

A. Kosta, N. Pappas, and V. Angelakis, Age of Information: A New Concept, Metric, and Tool, Foundations and Trends R in Networking, vol.12, issue.3, p.31, 2017.

H. Wang and N. Mandayam, A Simple Packet-Transmission Scheme for Wireless Data Over Fading Channels, IEEE Transactions on Communications, vol.52, issue.7, p.33, 2004.

Y. Chen and Q. Zhao, An Integrated Approach to Energy-Aware Medium Access for Wireless Sensor Networks, IEEE Transactions on Signal Processing, vol.55, issue.7, p.33, 2007.

D. V. Djonin and V. Krishnamurthy, MIMO Transmission Control in Fading Channels-A Constrained Markov Decision Process Formulation With Monotone Randomized Policies, IEEE Transactions on Signal Processing, vol.55, issue.10, p.33, 2007.

N. Salodkar, A. Bhorkar, A. Karandikar, and V. Borkar, An on-line learning algorithm for energy efficient delay constrained scheduling over a fading channel, IEEE Journal on Selected Areas in Communications, vol.26, p.33, 2008.

N. Salodkar, A. Karandikar, and V. S. Borkar, A Stable Online Algorithm for Energy-Efficient Multiuser Scheduling, IEEE Transactions on Mobile Computing, vol.9, issue.10, p.33, 2010.

R. A. Berry, Optimal Power-Delay Tradeoffs in Fading Channels-Small-Delay Asymptotics, IEEE Transactions on Information Theory, vol.59, issue.6, p.33, 2013.

K. T. Phan, T. Le-ngoc, M. Van-der-schaar, and F. Fu, Optimal Scheduling over Time-Varying Channels with Traffic Admission Control: Structural Results and Online Learning Algorithms, IEEE Transactions on Wireless Communications, vol.12, issue.9, p.33, 2013.

R. Berry and R. Gallager, Communication over fading channels with delay constraints, IEEE Transactions on Information Theory, vol.48, p.33, 2002.

A. Aprem, C. R. Murthy, and N. B. Mehta, Transmit Power Control Policies for Energy Harvesting Sensors With Retransmissions, IEEE Journal of Selected Topics in Signal Processing, vol.7, issue.5, pp.895-906, 2013.

Z. Xiang and M. Tao, Robust Beamforming for Wireless Information and Power Transmission, IEEE Wireless Communications Letters, vol.1, issue.4, p.63, 2012.

R. Gangula, D. Gesbert, and D. Gunduz, Optimization of Energy Harvesting MISO Communication System With Feedback, IEEE Journal on Selected Areas in Communications, vol.33, issue.3, p.64, 2015.

T. Le, G. Kaddoum, and O. Shin, Joint Channel Resources Allocation and Beamforming in Energy Harvesting Systems, IEEE Wireless Communications Letters, vol.7, issue.5, p.64, 2018.

R. Ma and W. Zhang, Optimal power allocation for energy harvesting communications with limited channel feedback, IEEE Global Conference on Signal and Information Processing, p.64, 2014.

M. R. Zenaidi, Z. Rezki, and M. Alouini, Performance Limits of Online Energy Harvesting Communications With Noisy Channel State Information at the Transmitter, IEEE Access, vol.5, p.64, 2017.

M. R. Zenaidi, Z. Rezki, H. Tembine, and M. Alouini, Performance limits of energy harvesting communications under imperfect channel state information, IEEE International Conference on Communications (ICC), p.64, 2016.

P. Ciblat and L. Vandendorpe, On the Maximum-Likelihood based data-aided frequency offset and channel estimates, European Signal Processing Conference (EUSIPCO), p.67, 2002.

J. G. Proakis and M. Salehi, Digital communications. Boston: McGraw-Hill, p.67, 2008.

S. Lin and D. J. Costello, Error control coding: fundamentals and applications, 2004.

A. Adhikary, H. C. Papadopoulos, S. A. Ramprashad, and G. Caire, Multi-user MIMO with outdated CSI: Training, feedback and scheduling, IEEE Annual Allerton Conference on Communication, Control, and Computing (Allerton), p.79, 2011.

W. Xiong, A. Mukherjee, and H. M. Kwon, Underlay MIMO Cognitive Radio Downlink Scheduling with Multiple Primary Users and No CSI, IEEE 81st Vehicular Technology Conference (VTC Spring), p.79, 2015.

M. J. Abdoli and A. S. Avestimehr, Layered Interference Networks With Delayed CSI: DoF Scaling With Distributed Transmitters, IEEE Transactions on Information Theory, vol.60, issue.3, p.79, 2014.

Y. Huang, S. Durrani, and X. Zhou, Interference nulling for offloaded heterogeneous users using macro generalized inverse precoder, IEEE International Symposium on Communications and Information Technologies (ISCIT), p.80, 2015.

P. Swami, V. Bhatia, S. Vuppala, and T. Ratnarajah, Performance Analysis of Offloading in NOMA-HetNets using Imperfect CSI, IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS), p.80, 2018.

L. Jiao, H. Yin, Y. Lyu, H. Huang, J. Dong et al., Optimal Schedule of Mobile Edge Computing Under Imperfect CSI, Algorithms and Architectures for Parallel Processing, vol.11335, p.80, 2018.

T. T. Nguyen, L. Le, and Q. Le-trung, Computation Offloading in MIMO Based Mobile Edge Computing Systems Under Perfect and Imperfect CSI Estimation, IEEE Transactions on Services Computing, p.80, 2019.

R. Atallah, C. Assi, and M. Khabbaz, Deep reinforcement learning-based scheduling for roadside communication networks, IEEE International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), p.92, 2017.

H. Ye and G. Y. Li, Deep Reinforcement Learning for Resource Allocation in V2v Communications, IEEE International Conference on Communications (ICC), p.92, 2018.

X. Chen, H. Zhang, C. Wu, S. Mao, Y. Ji et al., Performance Optimization in Mobile-Edge Computing via Deep Reinforcement Learning, p.92, 2018.

L. Huang, X. Feng, A. Feng, Y. Huang, and L. P. Qian, Distributed Deep Learning-based Offloading for Mobile Edge Computing Networks, Mobile Networks and Applications, p.92, 2018.

M. K. Sharma, A. Zappone, M. Debbah, and M. Assaad, Deep Learning Based Online Power Control for Large Energy Harvesting Networks, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p.92, 2019.

M. K. Sharma, A. Zappone, M. Assaad, M. Debbah, and S. Vassilaras, Distributed Power Control for Large Energy Harvesting Networks: A Multi-Agent Deep Reinforcement Learning Approach, p.92, 2019.

D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, p.93, 2014.

V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou et al., Playing Atari with Deep Reinforcement Learning, 2013.

N. C. Luong, D. T. Hoang, S. Gong, D. Niyato, P. Wang et al., Applications of Deep Reinforcement Learning in Communications and Networking: A Survey, p.95, 2018.

L. Dai, B. Wang, Z. Ding, Z. Wang, S. Chen et al., A Survey of Non-Orthogonal Multiple Access for 5g, IEEE Communications Surveys & Tutorials, vol.20, issue.3, p.96, 2018.

M. Aldababsa, M. Toka, S. Gökçeli, G. K. Kurt, and O. Kucur, A Tutorial on Nonorthogonal Multiple Access for 5g and Beyond, Wireless Communications and Mobile Computing, vol.2018, p.96, 2018.

P. Whittle, Restless bandits: activity allocation in a changing world, Journal of Applied Probability, vol.25, p.97, 1988.

I. Kadota, A. Sinha, E. Uysal-biyikoglu, E. Singh, and E. Modiano, Scheduling policies for minimizing age of information in broadcast wireless networks, IEEE/ACM Transactions on Networking, vol.26, issue.6, p.97, 2018.