Задача спарсить страницу url_get, но для этого необходима авторизация на mos.ru. POST запрос на авторизацию выглядит так:
Host: login.mos.ru
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate, br
Content-Type: application/x-www-form-urlencoded
Content-Length: 177
Origin: https://login.mos.ru
Connection: keep-alive
Referer: https://login.mos.ru/sps/login/methods/password
Cookie: fm=eyJtZXRob2RzIjpbInBhc3N3b3JkIiwieDUwOSIsImV4dGVybmFsSWRwcyJdLCJwYXJhbXMiOnsiY2hhbGxlbmdlIjoibW1HdkRYa1hvL1VXZ2lCWFdBVGMiLCJmZWRQb2ludHMiOiJ5YW5kZXg6eWFuZGV4XzF8c2JyZjpzYnJmXzF8Z29vZ2xlOmdvb2dsZV8xfHZrOnZrXzF8ZmFjZWJvb2s6ZmFjZWJvb2tfMXxlc2lhOmVzaWFfMXxvazpva18xIn19; blg=ru; history=eyJtZXRob2QiOiJwYXNzd29yZCJ9; origin=53w5fwZVaZiNYy4QYHZp|%2Fsps%2Foauth%2Fae%3Fscope%3Dprofile%2Bopenid%2Bcontacts%26response_type%3Dcode%26redirect_uri%3Dhttps%3A%2F%2Fwww.mos.ru%2Fapi%2Facs%2Fv1%2Flogin%2Fsatisfy%26client_id%3D53w5fwZVaZiNYy4QYHZp; oauth_az=GSpw72qzeKoBYs34Am9A5H8YsYa7ig6QN_xeE8ca-j_8VN9mDADOpewZVEJAbcyYkXnt53fTyed4XuYJyupsMoUQe9TcfePFGIoJLIgZRJ8; lstate=8hIGwhYxF2jCl2_Ylmnsho8-A_pEqQ4lxY7U2U0UFsLUtBf7BCHFGGzZn1S6GYB6O_nfTRJcE1WHgRmjmVe_Cc5o0rd6BAByYu2aWS7YBj_6M1hK4ihVRmEBYbptNK8UkaMn4qU_Z5v0WenyXCPhr4GXAM47uSy_5a6uG948gM94VzZ37J_mYmntW-n-yXx7exZOetqDwhqxML48NxJrxFwqrqVZFt5OYRgM8-0OxvIMR09GrKTG0BPGhGerZChzh8yjWxDFUfE_ozzzwXr57itKi4GlVFXIHrjkBYxVY3S5X022VqH068FluqQOESvCpO14r6qLk33slQ7H1-zrwLCNEhHNbdU3DYvgd6RoKbPDSgVSp-TqZEBwx7TpltJI-a5oJbESzA9Vu5ZVw9h2v7hzNaJy97ImW9VuvXRVBHwmIi_cN6SnJOWq49ypOcOE3u4iiuzgb6K_pz7NDa8f6JUJbmCpsZwJ2pCTMJ4cQsX1Peg09U95C75b026QpLyShsoMReUubbXtn2m9eYhvIfzgRswQMr8Sj4LgqH9BEiL3TZaIIMxyo-bagNaFRXRi|MTU4NTU1NzMzOQ|U0gxQVMxMjhDQkM|ER7HYKGdWR7sn-Pq3KmUkQ|mVjnsvm1VMcEiFkuHm2WVw44TVg; mos_id=Cg8qAl6BJ1oWGVI8hI3RAgA=; session-cookie=1600e85627f68f9920f90eb004983c473c883880c61917c6ccff2032d7fef65a68736013a1bf70cde27eb74cb7044e91; _ym_uid=1585522529585213091; _ym_d=1585522529; _ym_isad=2; _ym_visorc_32628510=w; _ym_visorc_14112952=b; csrf-token-name=lcsrftoken; csrf-token-value=1601080c203f92df03b4aa6069781475b7ae63ee44069f88a39007ed388a13475d6ba770aa19a068; _ym_visorc_53480143=w
Upgrade-Insecure-Requests: 1
isDelayed=false&login=12&password=12&lcsrftoken=160107fff2ad7b1347fd1db484bfe4e7aaeecc1a04fb96230405c09e04822e9d0a660a3ae244cf48&bfp=4ffc3baee892f5091ddbe8e625c2596a&alien=false
Пытался реализовать это так, но возвращает time out:
import requests
url_auth = 'https://login.mos.ru'
url_auth_ref = 'https://login.mos.ru/sps/login/methods/password'
url_get = 'https://www.mos.ru/visit/gibddlicense/reserve/?reasonId=26'
login = ''
password = ''
s = requests.Session()
r = s.get(url_auth_ref)
headers = {'Referer': url_auth_ref}
csrf_token = r.cookies['lcsrftoken']
data = {'login':login, 'password': password, 'lcsrftoken': csrf_token}
d = s.post(url_auth, data=data, headers=headers)
dd = s.get(url_get)
print(dd.text)```
KeyError: "name='lcsrftoken', domain=None, path=None"
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='login.mos.ru', port=443): Max retries exceeded with url: /sps/login/methods/password (Caused by <class 'TimeoutE or'>: [Errno 110] Connection timed out)
Но вероятнее всего ошибка в неверном задании cookiecsrf_token = r.cookies['lcsrftoken']