如何获取网络图的javascript标记中的数据,以便制作自己的图
How can I get the data inside a javascript tag for a web graph, so I can make my own graph?
到目前为止,使用python库twill和beautifulsoup,我能够登录到我的公用事业公司的网站,并分别访问包含我的天然气和电力使用数据的页面。实际上,他们用javascript绘制了一个很好的图,原始数据就埋在那里。
twill库非常好地显示了使用showforms()
、show()
等的web表单#1、#2、#3等的组成,但据我所知,还没有显示javascript元素beautifulsoup似乎能够显示html元素,比如网页的标题,例如:title1 = soup1.html.head.title
,但我似乎无法让它列出javascript元素(页面上大约有4或5个不同的java脚本,其中一个包含以下数据)。
有没有办法分别在twill或beautifulsoup中使用类似showjavascript()
或soup1.html.body.javascript
的东西来访问这些javascript数据,并可能对其进行解析以导入matplotlib或者其他python绘图包?
javascript元素中的数据如下(您可以看到我、邻居和高效邻居):
<script type="text/javascript">
(function() {
window.seriesDTO = {
"series" : [ {
"id" : "YOU",
"data" : [ {
"startDate" : "Fri, 30 Aug 2013 00:00:00",
"endDate" : "Mon, 23 Sep 2013 23:59:59",
"value" : 66,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sun, 01 Sep 2013 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 30 Aug 2013 00:00:00",
"upperDate" : "Mon, 23 Sep 2013 23:59:59"
}, {
"startDate" : "Tue, 24 Sep 2013 00:00:00",
"endDate" : "Tue, 22 Oct 2013 23:59:59",
"value" : 85,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Tue, 01 Oct 2013 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Tue, 24 Sep 2013 00:00:00",
"upperDate" : "Tue, 22 Oct 2013 23:59:59"
}, {
"startDate" : "Wed, 23 Oct 2013 00:00:00",
"endDate" : "Thu, 21 Nov 2013 23:59:59",
"value" : 135,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Fri, 01 Nov 2013 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Wed, 23 Oct 2013 00:00:00",
"upperDate" : "Thu, 21 Nov 2013 23:59:59"
}, {
"startDate" : "Fri, 22 Nov 2013 00:00:00",
"endDate" : "Mon, 23 Dec 2013 23:59:59",
"value" : 134,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sun, 01 Dec 2013 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 22 Nov 2013 00:00:00",
"upperDate" : "Mon, 23 Dec 2013 23:59:59"
}, {
"startDate" : "Tue, 24 Dec 2013 00:00:00",
"endDate" : "Thu, 23 Jan 2014 23:59:59",
"value" : 72,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Wed, 01 Jan 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Tue, 24 Dec 2013 00:00:00",
"upperDate" : "Thu, 23 Jan 2014 23:59:59"
}, {
"startDate" : "Fri, 24 Jan 2014 00:00:00",
"endDate" : "Mon, 24 Feb 2014 23:59:59",
"value" : 109,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sat, 01 Feb 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 24 Jan 2014 00:00:00",
"upperDate" : "Mon, 24 Feb 2014 23:59:59"
}, {
"startDate" : "Tue, 25 Feb 2014 00:00:00",
"endDate" : "Tue, 25 Mar 2014 23:59:59",
"value" : 84,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sat, 01 Mar 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Tue, 25 Feb 2014 00:00:00",
"upperDate" : "Tue, 25 Mar 2014 23:59:59"
}, {
"startDate" : "Wed, 26 Mar 2014 00:00:00",
"endDate" : "Thu, 24 Apr 2014 23:59:59",
"value" : 86,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Tue, 01 Apr 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Wed, 26 Mar 2014 00:00:00",
"upperDate" : "Thu, 24 Apr 2014 23:59:59"
}, {
"startDate" : "Fri, 25 Apr 2014 00:00:00",
"endDate" : "Fri, 23 May 2014 23:59:59",
"value" : 56,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Thu, 01 May 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 25 Apr 2014 00:00:00",
"upperDate" : "Fri, 23 May 2014 23:59:59"
}, {
"startDate" : "Sat, 24 May 2014 00:00:00",
"endDate" : "Tue, 24 Jun 2014 23:59:59",
"value" : 76,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sun, 01 Jun 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Sat, 24 May 2014 00:00:00",
"upperDate" : "Tue, 24 Jun 2014 23:59:59"
}, {
"startDate" : "Wed, 25 Jun 2014 00:00:00",
"endDate" : "Thu, 24 Jul 2014 23:59:59",
"value" : 72,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Tue, 01 Jul 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Wed, 25 Jun 2014 00:00:00",
"upperDate" : "Thu, 24 Jul 2014 23:59:59"
}, {
"startDate" : "Fri, 25 Jul 2014 00:00:00",
"endDate" : "Mon, 25 Aug 2014 23:59:59",
"value" : 81,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Fri, 01 Aug 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 25 Jul 2014 00:00:00",
"upperDate" : "Mon, 25 Aug 2014 23:59:59"
} ],
"unit" : "KWH",
"precision" : 0
}, {
"id" : "NEIGHBORS",
"data" : [ {
"startDate" : "Fri, 30 Aug 2013 00:00:00",
"endDate" : "Mon, 23 Sep 2013 23:59:59",
"value" : 257.958526611328125,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sun, 01 Sep 2013 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 30 Aug 2013 00:00:00",
"upperDate" : "Mon, 23 Sep 2013 23:59:59"
}, {
"startDate" : "Tue, 24 Sep 2013 00:00:00",
"endDate" : "Tue, 22 Oct 2013 23:59:59",
"value" : 316.710968017578125,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Tue, 01 Oct 2013 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Tue, 24 Sep 2013 00:00:00",
"upperDate" : "Tue, 22 Oct 2013 23:59:59"
}, {
"startDate" : "Wed, 23 Oct 2013 00:00:00",
"endDate" : "Thu, 21 Nov 2013 23:59:59",
"value" : 373.00909423828125,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Fri, 01 Nov 2013 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Wed, 23 Oct 2013 00:00:00",
"upperDate" : "Thu, 21 Nov 2013 23:59:59"
}, {
"startDate" : "Fri, 22 Nov 2013 00:00:00",
"endDate" : "Mon, 23 Dec 2013 23:59:59",
"value" : 442.25927734375,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sun, 01 Dec 2013 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 22 Nov 2013 00:00:00",
"upperDate" : "Mon, 23 Dec 2013 23:59:59"
}, {
"startDate" : "Tue, 24 Dec 2013 00:00:00",
"endDate" : "Thu, 23 Jan 2014 23:59:59",
"value" : 389.46453857421875,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Wed, 01 Jan 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Tue, 24 Dec 2013 00:00:00",
"upperDate" : "Thu, 23 Jan 2014 23:59:59"
}, {
"startDate" : "Fri, 24 Jan 2014 00:00:00",
"endDate" : "Mon, 24 Feb 2014 23:59:59",
"value" : 402.319488525390625,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sat, 01 Feb 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 24 Jan 2014 00:00:00",
"upperDate" : "Mon, 24 Feb 2014 23:59:59"
}, {
"startDate" : "Tue, 25 Feb 2014 00:00:00",
"endDate" : "Tue, 25 Mar 2014 23:59:59",
"value" : 319.855438232421875,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sat, 01 Mar 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Tue, 25 Feb 2014 00:00:00",
"upperDate" : "Tue, 25 Mar 2014 23:59:59"
}, {
"startDate" : "Wed, 26 Mar 2014 00:00:00",
"endDate" : "Thu, 24 Apr 2014 23:59:59",
"value" : 328.7225341796875,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Tue, 01 Apr 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Wed, 26 Mar 2014 00:00:00",
"upperDate" : "Thu, 24 Apr 2014 23:59:59"
}, {
"startDate" : "Fri, 25 Apr 2014 00:00:00",
"endDate" : "Fri, 23 May 2014 23:59:59",
"value" : 301.809600830078125,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Thu, 01 May 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 25 Apr 2014 00:00:00",
"upperDate" : "Fri, 23 May 2014 23:59:59"
}, {
"startDate" : "Sat, 24 May 2014 00:00:00",
"endDate" : "Tue, 24 Jun 2014 23:59:59",
"value" : 324.0477294921875,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sun, 01 Jun 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Sat, 24 May 2014 00:00:00",
"upperDate" : "Tue, 24 Jun 2014 23:59:59"
}, {
"startDate" : "Wed, 25 Jun 2014 00:00:00",
"endDate" : "Thu, 24 Jul 2014 23:59:59",
"value" : 291.14556884765625,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Tue, 01 Jul 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Wed, 25 Jun 2014 00:00:00",
"upperDate" : "Thu, 24 Jul 2014 23:59:59"
}, {
"startDate" : "Fri, 25 Jul 2014 00:00:00",
"endDate" : "Mon, 25 Aug 2014 23:59:59",
"value" : 326.228668212890625,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Fri, 01 Aug 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 25 Jul 2014 00:00:00",
"upperDate" : "Mon, 25 Aug 2014 23:59:59"
} ],
"unit" : "KWH",
"precision" : 0
}, {
"id" : "EFFICIENT_NEIGHBORS",
"data" : [ {
"startDate" : "Fri, 30 Aug 2013 00:00:00",
"endDate" : "Mon, 23 Sep 2013 23:59:59",
"value" : 87.5,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sun, 01 Sep 2013 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 30 Aug 2013 00:00:00",
"upperDate" : "Mon, 23 Sep 2013 23:59:59"
}, {
"startDate" : "Tue, 24 Sep 2013 00:00:00",
"endDate" : "Tue, 22 Oct 2013 23:59:59",
"value" : 112.07543182373046875,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Tue, 01 Oct 2013 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Tue, 24 Sep 2013 00:00:00",
"upperDate" : "Tue, 22 Oct 2013 23:59:59"
}, {
"startDate" : "Wed, 23 Oct 2013 00:00:00",
"endDate" : "Thu, 21 Nov 2013 23:59:59",
"value" : 123,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Fri, 01 Nov 2013 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Wed, 23 Oct 2013 00:00:00",
"upperDate" : "Thu, 21 Nov 2013 23:59:59"
}, {
"startDate" : "Fri, 22 Nov 2013 00:00:00",
"endDate" : "Mon, 23 Dec 2013 23:59:59",
"value" : 143,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sun, 01 Dec 2013 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 22 Nov 2013 00:00:00",
"upperDate" : "Mon, 23 Dec 2013 23:59:59"
}, {
"startDate" : "Tue, 24 Dec 2013 00:00:00",
"endDate" : "Thu, 23 Jan 2014 23:59:59",
"value" : 116,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Wed, 01 Jan 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Tue, 24 Dec 2013 00:00:00",
"upperDate" : "Thu, 23 Jan 2014 23:59:59"
}, {
"startDate" : "Fri, 24 Jan 2014 00:00:00",
"endDate" : "Mon, 24 Feb 2014 23:59:59",
"value" : 131,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sat, 01 Feb 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 24 Jan 2014 00:00:00",
"upperDate" : "Mon, 24 Feb 2014 23:59:59"
}, {
"startDate" : "Tue, 25 Feb 2014 00:00:00",
"endDate" : "Tue, 25 Mar 2014 23:59:59",
"value" : 109,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sat, 01 Mar 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Tue, 25 Feb 2014 00:00:00",
"upperDate" : "Tue, 25 Mar 2014 23:59:59"
}, {
"startDate" : "Wed, 26 Mar 2014 00:00:00",
"endDate" : "Thu, 24 Apr 2014 23:59:59",
"value" : 105,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Tue, 01 Apr 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Wed, 26 Mar 2014 00:00:00",
"upperDate" : "Thu, 24 Apr 2014 23:59:59"
}, {
"startDate" : "Fri, 25 Apr 2014 00:00:00",
"endDate" : "Fri, 23 May 2014 23:59:59",
"value" : 103,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Thu, 01 May 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 25 Apr 2014 00:00:00",
"upperDate" : "Fri, 23 May 2014 23:59:59"
}, {
"startDate" : "Sat, 24 May 2014 00:00:00",
"endDate" : "Tue, 24 Jun 2014 23:59:59",
"value" : 119,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Sun, 01 Jun 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Sat, 24 May 2014 00:00:00",
"upperDate" : "Tue, 24 Jun 2014 23:59:59"
}, {
"startDate" : "Wed, 25 Jun 2014 00:00:00",
"endDate" : "Thu, 24 Jul 2014 23:59:59",
"value" : 102,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Tue, 01 Jul 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Wed, 25 Jun 2014 00:00:00",
"upperDate" : "Thu, 24 Jul 2014 23:59:59"
}, {
"startDate" : "Fri, 25 Jul 2014 00:00:00",
"endDate" : "Mon, 25 Aug 2014 23:59:59",
"value" : 104,
"estimated" : false,
"shouldUseMajorityMonthForBilling" : true,
"startOfMajorityMonth" : "Fri, 01 Aug 2014 00:00:00",
"firstMajorityMonth" : null,
"lastMajorityMonth" : null,
"lowerDate" : "Fri, 25 Jul 2014 00:00:00",
"upperDate" : "Mon, 25 Aug 2014 23:59:59"
} ],
"unit" : "KWH",
"precision" : 0
}, {
"id" : "BILL_IDENTIFIERS",
"data" : [ {
"value" : 1,
"year" : 2013,
"startDate" : "Fri, 30 Aug 2013 00:00:00",
"endDate" : "Mon, 23 Sep 2013 23:59:59",
"lowerDate" : "Fri, 30 Aug 2013 00:00:00",
"upperDate" : "Mon, 23 Sep 2013 23:59:59"
}, {
"value" : 2,
"year" : 2013,
"startDate" : "Tue, 24 Sep 2013 00:00:00",
"endDate" : "Tue, 22 Oct 2013 23:59:59",
"lowerDate" : "Tue, 24 Sep 2013 00:00:00",
"upperDate" : "Tue, 22 Oct 2013 23:59:59"
}, {
"value" : 3,
"year" : 2013,
"startDate" : "Wed, 23 Oct 2013 00:00:00",
"endDate" : "Thu, 21 Nov 2013 23:59:59",
"lowerDate" : "Wed, 23 Oct 2013 00:00:00",
"upperDate" : "Thu, 21 Nov 2013 23:59:59"
}, {
"value" : 4,
"year" : 2013,
"startDate" : "Fri, 22 Nov 2013 00:00:00",
"endDate" : "Mon, 23 Dec 2013 23:59:59",
"lowerDate" : "Fri, 22 Nov 2013 00:00:00",
"upperDate" : "Mon, 23 Dec 2013 23:59:59"
}, {
"value" : 1,
"year" : 2014,
"startDate" : "Tue, 24 Dec 2013 00:00:00",
"endDate" : "Thu, 23 Jan 2014 23:59:59",
"lowerDate" : "Tue, 24 Dec 2013 00:00:00",
"upperDate" : "Thu, 23 Jan 2014 23:59:59"
}, {
"value" : 2,
"year" : 2014,
"startDate" : "Fri, 24 Jan 2014 00:00:00",
"endDate" : "Mon, 24 Feb 2014 23:59:59",
"lowerDate" : "Fri, 24 Jan 2014 00:00:00",
"upperDate" : "Mon, 24 Feb 2014 23:59:59"
}, {
"value" : 3,
"year" : 2014,
"startDate" : "Tue, 25 Feb 2014 00:00:00",
"endDate" : "Tue, 25 Mar 2014 23:59:59",
"lowerDate" : "Tue, 25 Feb 2014 00:00:00",
"upperDate" : "Tue, 25 Mar 2014 23:59:59"
}, {
"value" : 4,
"year" : 2014,
"startDate" : "Wed, 26 Mar 2014 00:00:00",
"endDate" : "Thu, 24 Apr 2014 23:59:59",
"lowerDate" : "Wed, 26 Mar 2014 00:00:00",
"upperDate" : "Thu, 24 Apr 2014 23:59:59"
}, {
"value" : 5,
"year" : 2014,
"startDate" : "Fri, 25 Apr 2014 00:00:00",
"endDate" : "Fri, 23 May 2014 23:59:59",
"lowerDate" : "Fri, 25 Apr 2014 00:00:00",
"upperDate" : "Fri, 23 May 2014 23:59:59"
}, {
"value" : 6,
"year" : 2014,
"startDate" : "Sat, 24 May 2014 00:00:00",
"endDate" : "Tue, 24 Jun 2014 23:59:59",
"lowerDate" : "Sat, 24 May 2014 00:00:00",
"upperDate" : "Tue, 24 Jun 2014 23:59:59"
}, {
"value" : 7,
"year" : 2014,
"startDate" : "Wed, 25 Jun 2014 00:00:00",
"endDate" : "Thu, 24 Jul 2014 23:59:59",
"lowerDate" : "Wed, 25 Jun 2014 00:00:00",
"upperDate" : "Thu, 24 Jul 2014 23:59:59"
}, {
"value" : 8,
"year" : 2014,
"startDate" : "Fri, 25 Jul 2014 00:00:00",
"endDate" : "Mon, 25 Aug 2014 23:59:59",
"lowerDate" : "Fri, 25 Jul 2014 00:00:00",
"upperDate" : "Mon, 25 Aug 2014 23:59:59"
} ]
} ],
"sufficientData" : true
};
})();
</script>
您可以使用slimit
包解析javascript代码:
SlimIt是一个用Python编写的JavaScript迷你程序。它编译将JavaScript转换为更紧凑的代码,以便下载和运行更快。
SlimIt还提供了一个库,该库包括JavaScript解析器,lexer,漂亮的打印机和一个树访客。
在您的情况下,您可以通过找到对象的第一个赋值来应用它:
import json
from bs4 import BeautifulSoup
from slimit import ast
from slimit.parser import Parser
from slimit.visitors import nodevisitor
data = """
<script type="text/javascript">
(function() {
window.seriesDTO = {
...
};
})();
</script>
"""
soup = BeautifulSoup(data)
parser = Parser()
tree = parser.parse(soup.script.text)
json_data = next(node.right.to_ecma() for node in nodevisitor.visit(tree)
if isinstance(node, ast.Assign) and isinstance(node.right, ast.Object))
print json.loads(json_data)
打印Python字典:
{u'series': [{u'data': [{u'endDate': u'Mon, 23 Sep 2013 23:59:59',
u'estimated': False,
u'firstMajorityMonth': None,
u'lastMajorityMonth': None,
u'lowerDate': u'Fri, 30 Aug 2013 00:00:00',
u'shouldUseMajorityMonthForBilling': True,
u'startDate': u'Fri, 30 Aug 2013 00:00:00',
u'startOfMajorityMonth': u'Sun, 01 Sep 2013 00:00:00',
u'upperDate': u'Mon, 23 Sep 2013 23:59:59',
u'value': 66},
...
{u'endDate': u'Mon, 25 Aug 2014 23:59:59',
u'lowerDate': u'Fri, 25 Jul 2014 00:00:00',
u'startDate': u'Fri, 25 Jul 2014 00:00:00',
u'upperDate': u'Mon, 25 Aug 2014 23:59:59',
u'value': 8,
u'year': 2014}],
u'id': u'BILL_IDENTIFIERS'}],
u'sufficientData': True}
相关文章:
- 如何配置分析以将数据发送到我自己的服务器
- 如何使用jquery ajax和jsonp在您自己的域上读取json数据
- jQuery:仅显示新的JSON数据,并使用自己的段落标签进行迭代
- 我该如何将自己的数据添加到响应迅速的美国各州地图中
- 将json数据发送到另一个url-在您'我从我自己的网址上得到的
- PubNub或Pusher,并将数据存储在我自己的服务器上
- WebRTC是否为您处理数据包丢失和数据包接收确认等,或者您是否需要为此编写自己的解决方案
- 谷歌可视化多个图表与自己的数据查询
- MySQL如何更新自己的数据
- 以我自己的格式创建随机数据
- 挖空添加我自己的数据绑定属性,如 With
- 用自己的数据更新流星排行榜
- Google Analytics Outbound Click数据室(在我自己的服务器中)
- 如何访问元素's使用jQuery在其自己的事件处理程序中的数据
- Google Analytics、GoSquared和我自己的统计数据之间存在巨大的页面浏览量差异
- 如何为拥有它的HTML片段提供初始数据'自己的控制器
- 如何为我自己的搜索引擎获取初始索引数据
- Meteor发布:管理员可以查看所有数据,用户只能查看自己的数据
- 如何获取网络图的javascript标记中的数据,以便制作自己的图
- 使用自己的Facebook访问令牌查询数据以填充数据库