Table not found error after submitting a spark script consisting with spark sql after enabling hivesupport

Uncategorized
Question: I want to run a simple spark script which has some sparksql query basicaly Hiveql. the corresponding tables are saved in spark-warehouse folder. from pyspark.sql import SparkSession from pyspark.sql import Row spark=SparkSession.builder.config("spark.sql.warehouse.dir", "file:///C:/tmp").appName("TestApp").enableHiveSupport().getOrCreate() sqlstring="SELECT lflow1.LeaseType as LeaseType, lflow1.Status as Status, lflow1.Property as property, lflow1.City as City, lesflow2.DealType as DealType, lesflow2.Area as Area, lflow1.Did as DID, lesflow2.MID as MID from lflow1, lesflow2 WHERE lflow1.Did = lesflow2.MID" def queryBuilder(sqlval):df=spark.sql(sqlval) df.show() result=queryBuilder(sqlstring) print(result.collect()) print("Type of",type(result)) after performing spark submit operation i am facing bellow error py4j.protocol.Py4JJavaError: An error occurred while calling o27.sql. : org.apache.spark.sql.AnalysisException: Table or view not found: lflow1; line 1 pos 211 I could not figure out why it is happening. I have seen some posts from stackoverflow and they suggested that i have to enable my hive support which…
Read More

Scope of any element/object of a response json is within the scenario only

Uncategorized
Question: Scenario: Generenate jwt and check status Given path '/sdk/jwt' header Authorization = call read('jwt.js') { token: 'e68c82a665847c', secret: 'f08f06f1f41f4479854c' } When method get Then status 200 And def tkn = response Scenario: Get project meta info for an instance Given path '/meta/project' And header Authorization = JWT tkn #which the response of 1st scenario. When method get Then status 200 But tkn value is not coming in the second scenario. Answer: Read more
Read More

Leezen – Health & Yoga WordPress Theme

Uncategorized
Leezen is our new WordPress theme intended to serve as yoga school, retreat center, holistic center, therapist’s, body treatment salon, wellness and spa center, or beauty shop’s website. The theme offers an easy manageable homepage layout that can be adjusted using shortcodes and custom widgets. Attractive galleries integrated due to Essential Grid allow you to showcase your products and services.
Read More

Xamarin App is not opening in Emulators in windows 8.1 pro

c#
Question: Started developing a cross platform xamarin app template, and selected xamarin forms and portable native library , and then when I start debugging with android app, then emulator selected is not able to run the app. Its only showing the black screen , but able to capture screenshot . So how can I have more emulator option for windows 8.1 pro 32 bit system. I also tried to install the android emulator for windows, but available with 64 bit configuration system. Its not supported by windows 8.1 pro 32 bit system. Please suggest some possible solutions. Answer: Read more
Read More

Insert DOM element into React component

Uncategorized
Question: I am using django as backend and most of it as frontend as well, the exception is one form that is made in reactjs. To speed up loading of this form I would like to load it's data by grabing it from html instead of using request. Django would send the data and form on loading the page and another request would not be neccessary. I managed to grab the data, which is html, but I am running in to following error: Uncaught (in promise) Error: Objects are not valid as a React child (found: [object HTMLOptionElement]). If you meant to render a collection of children, use an array instead or wrap the object using createFragment(object) from the React add-ons. Check the render method For testing purposes I am…
Read More

open activity from 2 different notification

Uncategorized
Question: i build notification in chat apps, and then i receive 2 notification from userA and userB. When i click notif from userA, its open ConversationActivity between me and userA. But while conversation between me and userA still open i click notif from userB, and then what i've got is still in conversation between me and userA not with userB. This is when i build notif private NotificationCompat.Builder createTypeNotif(String message, GetContactResponse contact){ NotificationCompat.Builder builder = new NotificationCompat.Builder(MessagingService.this); String[] parts = splitMessage(message); if (parts.length > 1){ if (parts[0].equals(MessagingService.CHAT_PROTOCOL) || parts[0].equals(MessagingService.LOC_PROTOCOL)){ resultIntent = new Intent(MessagingService.this, ConversationActivity.class); resultIntent.setFlags(Intent.FLAG_ACTIVITY_CLEAR_TOP | Intent.FLAG_ACTIVITY_SINGLE_TOP); resultIntent.putExtra(ConversationActivity.class.getSimpleName() + ".empId", contact.getEmpId()); resultIntent.putExtra(ConversationActivity.class.getSimpleName() + ".message", message); builder.setSmallIcon(R.drawable.ic_notif_chat) .setContentTitle("Incoming Message") .setContentText("You got message from " + contact.getName()); builder.setLargeIcon(BitmapFactory.decodeResource(getResources(),R.mipmap.ic_launcher)); return builder; }else{ return builder; } }else{ return builder; } } Answer: use this…
Read More

postgres localhost SQL state: 42501 file permissions on mac osx sierra to import csv files

Uncategorized
Question: Upon trying to import csv files into a table in my database (localhost) using both sql and the csv import wizard in postgres I get the following error:ERROR: could not open file "...somefile.csv" for reading: Permission denied SQL state: 42501 What I have tried I have given the folders where the csv data is read/write access for the default postgres user using mac Get Info dialog. COPY stockprices (OpenPrice integer, High integer, Low integer, ClosePrice integer, Adj_Close integer, Volume integer, Stock text, Date date) from '/Users/myuser/Documents/myfile.csv' DELIMITER ',' CSV HEADER; References No Luck with this answer Answer: Read more
Read More

spark onQueryProgress duplicate

Uncategorized
Question: when i monitor structured streaming use StreamingQueryListener ,I found duplicate in on onQueryProgress override def onQueryProgress(queryProgress: QueryProgressEvent): Unit = { if(queryProgress.progress.numInputRows!=0) { println("Query made progress: " + queryProgress.progress) } the result is Query made progress: { "id" : "e76a8789-738c-49f6-b7f4-d85356c28600", "runId" : "d8ce0fad-db38-4566-9198-90169efeb2d8", "name" : null, "timestamp" : "2017-08-15T07:28:27.077Z", "numInputRows" : 1, "processedRowsPerSecond" : 0.3050640634533252, "durationMs" : { "addBatch" : 2452, "getBatch" : 461, "queryPlanning" : 276, "triggerExecution" : 3278 }, "stateOperators" : [ ], "sources" : [ { "description" : "KafkaSource[Subscribe[test1]]", "startOffset" : { "test1" : { "0" : 19 } }, "endOffset" : { "test1" : { "0" : 20 } }, "numInputRows" : 1, "processedRowsPerSecond" : 0.3050640634533252 } ], "sink" : { "description" : "org.apache.spark.sql.execution.streaming.ForeachSink@3ec8a100" } } Query made progress: { "id" : "a5b1f905-5575-43a7-afe9-dead0e4de2a7", "runId" : "8caea640-8772-4aab-ab13-84c1e952fb77",…
Read More

Why is my parsed JSON converting into a list python?

Uncategorized
Question: I am trying to parse a JSON file from an api, and i am not sure why it's converting to a list. from urllib2 import Request, urlopen import json Auth_key = 'get ur own key' headers = { 'Authorization': Auth_key } request = Request('https://app.com/orders?orderStatus=awaiting_shipment', headers=headers) def remove_nulls(d):d = {k: v for k, v in d.iteritems() if v is not None} return { k:v for k, v in d.items() if v } response_body = urlopen(request).read() parsed = json.loads(response_body, object_hook=remove_nulls) jsonData = parsed["orders"] #print(json.dumps(jsonData, indent=1, sort_keys=True)) print type(jsonData) When I check "parsed" it's a dict but "jsonData" is not. Thanks in advance Answer: You are accessing a key from your dict so it can be any other datatype. This might be a case with you: In [91]: jsonData = {'orders': [1,…
Read More

Example for using Guice with Endpoints Frameworks for App Engine version 2

java
Question: Does anyone know a good example for using Guice with the new Google Endpoints Frameworks for App Engine Version 2. In the Documentation Migrating to Endpoints Frameworks for App Engine Guice is mentioned at the end of the docu. But i am missing a full example with web.xml, appengine-web.xml, Module class and Eclipse. Answer: Add endpoints-framework-guice as a dependency. Add an instance of EndpointsModule to your GuiceFilter. Additionally bind GuiceFilter to /_ah/api/* in web.xml to work around a legacy URL resolution issue. Read more
Read More