Spark Filter data with groupby count

Uncategorized
Question: A dataframe A_df like:+------+----+-----+ | uid|year|month| +------+----+-----+ | 1|2017| 03| 1|2017| 05| | 2|2017| 01| | 3|2017| 02| | 3|2017| 04| | 3|2017| 05| +------+----+-----+ I want filter column uid with occurrence time more than 2 times, Expected result:+------+----+-----+ | uid|year|month| +------+----+-----+ | 3|2017| 02| | 3|2017| 04| | 3|2017| 05| +------+----+-----+ How can I get this result by scala? My solution:val condition_uid = A_df.groupBy("uid") .agg(count("*").alias("cnt")) .filter("cnt > 2").select("uid") val results_df = A_df.join(condition_uid, Seq("uid")) Is there better answer? Answer: I think using window function is the perfect solution since you do not have to rejoin the dataframe. val window = Window.partitionBy("uid").orderBy("year") df.withColumn("count", count("uid").over(window)) .filter($"count" > 2).drop("count").show Output: +---+----+-----+-----+ |uid|year|month|count| +---+----+-----+-----+ | 1|2017| 03| 2| | 1|2017| 05| 2| | 2|2017| 01| 1| +---+----+-----+-----+ Read more
Read More

how to debug npm packet written by c++

Uncategorized
Question: i wrote a node.js packet.When node load it,how to use gdb debug it. i usually use netbeans to debug C++. netbeans support c++ and node.js.But do not know how to debug such node.js packets written by C++. Answer: If I understand well, you want to debug an Addon Node.js (written in c++), this is not a module even if you can load it as a node-module. Unfortunately you cannot debug easily because there are 3 levels of code, the c++ code source, the v8(NaN) source which is a sugar above node & the node.js call of this add-on So you will not find an IDE to fully trace the road from node to c++ source code :s Read more
Read More

webpack-dev-server + browser-sync HMR says is get update – SOCKET HANDSHAKE ERROR

Uncategorized
Question: tinkering with webpack and associated stuff for the first time. i am trying to combine the benefits of webpack-dev-server and browser-sync to make it so when i update a react component my browser doesn't reload, it simply uses hot module replacement. there are plenty of boilerplates i guess, but i am starting from scratch. im to the point where HMR is on and listening for changes. i edit a component and it receives the change and says App is up to date. But the view doesn't change an a few seconds later i get a Web Socket error. if that is hard to read it basically says Web Socket error, socket closed before completing handshake... my webpack config const webpack = require("webpack"); const path = require("path"); const SRC_DIR =…
Read More

Cookie Not Saving To Another url with the same domain

Uncategorized
Question: This is the sample code of my project. I created an ajax function on my html file that will go to api_base_url/login?id=P1003 $("#trigger").click(function(){ $.ajax({ url : api_base_url + "/login", type : "GET", data : {id: "P1003"}, success : function (response){ if(response != null) { window.open(api_base_url + response); } } }); }) In my controller there is a GetMapping function for "/login" that will create cookies on the domain and will return a string URL on where he is going to redirect next. @RequestMapping("/login") public String login(HttpServletResponse response, @RequestParam(value = "id", defaultValue = "*") String id){ Cookie SESSION_ID = new Cookie("SESSION_ID", id); SESSION_ID.setPath("/"); response.addCookie(SESSION_ID); return "/home"; } Now I also have a controller for "/home" to get the cookies I set in the browser. @RequestMapping("/home") public ModelAndView home(HttpServletRequest request, HttpServletResponse…
Read More

Adding long click on parameters

Uncategorized
Question: I made this app from where a user can stream songs froma n artist. In building this app I used only onclick attribute all the time . but now I want to implement long click on each artist's name , so that a toast can popup about them. Can anyone help me in having longclick option . Thanks in advance. Answer: Simply on listview, add following code to get long click on each item of list view:listView.setOnItemLongClickListener(new AdapterView.OnItemLongClickListener() { @Override public boolean onItemLongClick(AdapterView parent, View view, int position, long id) { return false; } }); For Linear Layout, you may use this code_linearLayout= (LinearLayout)findViewById(R.id.linearLayout); linearLayout.setOnLongClickListener(new View.OnLongClickListener() { @Override public boolean onLongClick(View v) { return false; } }); Thanks, let me know if need more help. Read more
Read More

How to set up listener to salesforce outbound message in java using maven?

Uncategorized
Question: I am trying to set up salesforce outbound messaging listener in java. This is the link I followed to set up, and it works fine - https://developer.salesforce.com/page/Creating_an_Outbound_Messaging_Notification_Service_with_Eclipse_3.2 The article was unfortunately never updated after 2012, and I want to set this up as a maven project for obvious reasons -this is where I run into difficulties. I am looking for the right maven plugin to basically reproduce whatever was in the link.The idea is that, I need to have an endpoint listener, such that it can receive a soap message from salesforce. With me right now, I have a wsdl file (that I get from salesforce), and this is my starting point. The endpoint is also included in the wsdl. From a lot of research, I decided to use…
Read More